Deploying custom model in AWS Sagemaker

Photo by Markus Winkler on Unsplash

Introduction to AWS Sagemaker

AWS Sagemaker is an service provided by AWS for creating, training and deploying machine learning models. It was launched in 2017 to enable developers easily create models by creating an ecosystem for all services needed for the entire machine learning lifecycle.

Although Sagemaker is well equipped with many of the models for deployment, there still can be some models that are custom and Sagemaker will not have any Estimator(class used for creating an endpoint for deployment). In this article, we will take an example of a custom model and how it can be deployed in Sagemaker using a docker image.

Introduction to Docker

Docker is a container containers that provide virtualized runtime environments for running applications. It represents a package of software that contains code, system tools, runtime, libraries, dependencies, and configuration files required for running a specific application. They are independent and isolated from the host and other instances running on the host.

Introduction to ECR

ECR stands for Elastic Container Registry- a service provided by AWS for uploading and storing the image(docker) created locally. ECR creates a url that can be directly accessed in your Jupyter Notebook during model building process.

Creating a custom model docker image

Now that we covered the basics, we will now do an hands on exercise of creating a custom model image, uploading it to ECR and using it in a Sagemaker Notebook.

1. Install docker in your local machine- MAC, UBUNTU or WINDOWS
2. Download and save this Github repo in your local machine — https://github.com/aws/amazon-sagemaker-examples/tree/main/advanced_functionality/scikit_bring_your_own/container
3. Now we will build the docker image. Go to the folder location where the github repo is saved.
4. Enter the below command

docker build -t decision-tree .

5. This will build the image. You can validate by entering the below command-

docker images

6. Now you need to run the docker container to check the training script.

docker run --rm -v $(pwd)/local_test/test_dir:/opt/ml decision-tree train

7. Once this runs successfully, you would be able to see a message — Training Complete

8. Now we need to upload this image in ECR. Go to the AWS console — https://aws.amazon.com/console/

9. Search for ECR- Elastic Container Registry.

10. Now click on — Get Started”and then Click “Create repository

11. Name the repository whatever you want as follows. Click on create repository.

12. Now select the new repo and click “View push commands”

13. Follow the commands and do the same to push the image from your local to this empty repository you just created.

14. Now we are going to create a Sagemaker Notebook and use this image that we just created. Go to Sagemaker.

15. Create a new notebook instance (or use an existing one). You can use the default settings, and set a name.

16. Copy the following code in your notebook -

# Install sagemaker (version 1.72)

importsys!{

sys.executable} -m

pipinstall — quietsagemaker==1.72-U

# Imports

import io

import os

import sys

import time

import json

from IPython.display import display

from time import strftime,gmtime

import boto3

import re

import sagemaker

from sagemaker import get_execution_role

# Get the boto3 session and sagemaker client, as well as the current execution role

sess=boto3.Session()

sm=sess.client(‘sagemaker’)

role=sagemaker.get_execution_role()

# Name of the docker image containing the model code

docker_image_name=’<Name of docker image in registry on AWS>’

# Name and prefix for the S3 bucket storing the model outputaccount_id=sess.client(‘sts’, region_name=sess.region_name).get_caller_identity()[“Account”]bucket=’sagemaker-studio-{}-{}’.format(sess.region_name, account_id)prefix=’decision-tree’

This will complete the set up for your Sagemaker client, role and image repository.

Now we need to create an Estimator to train and deploy the model.

17. sess=sagemaker.session.Session()

decision-tree =

sagemaker.estimator.Estimator(image_name=docker_image_name,role=role,train_instance_count=1, train_instance_type=’ml.m4.xlarge’,output_path=’s3://{}/{}/output’.format(bucket, prefix),base_job_name=”decision-tree”,sagemaker_session=sess)

18. Next, we call fit to train the model. This runs the train script in the docker image model directory.

decision-tree.fit()

19. Finally,after training the mode, now we deploy the model as follows-

endpoint_name=’decision-tree-endpoint’

decision-tree.deploy(initial_instance_count=1, instance_type=’ml.t2.medium’, endpoint_name=endpoint_name)

# Delete endpoint when you don’t need your model deployed

sm.delete_endpoint(EndpointName=endpoint_name)

Now we have our custom model deployed! Hope you enjoyed the article. Please feel free to connect with me/follow me.