Adjustments for sagemaker implementation

ArneRobben commented 1 year ago

Description

This is a PR which aims to build a docker container which can be deployed to ECR and can be used by Sagemaker inference. The idea is to mimic the methodology used in this Sagemaker test repository.

I haven't finished setting everything up, it is still a work in progress but the main additions are:

a Dockerfile.sagemaker_inference dockerfile which aims to set up an image with grants_tagger + the latest model
a sagemaker_build_and_push.sh file which builds the image and pushes it to ECR
a folder sagemaker_inference which contains the additional bits of code needed for Sagemaker to interact with the image.

Currently I am still testing if I can upload the docker image to ECR and create an end point that can ping into it. The main code to do the actual tagging is task.py in sagemaker_inference, for which I have started to set up a @app.route("/invocations", methods=["POST"]) route, but I don't expect this to work straight away.

I'll keep posting my recent updates here until I go on parental leave..

Checklist

[ ] Linked to Notion or GitHub issue
[ ] Added tests
[ ] Updated README
[ ] DVC up to date

Release checklist

[ ] DVC repro up to date
[ ] Models synced in S3
[ ] Version updated
[ ] CHANGELOG.md updated
[ ] Model and package version aligned

To release:

make build
make deploy

nsorros commented 1 year ago

Is the latest model in dvc?

nsorros commented 1 year ago

I assume you are using some tutorial or guide for this but from what I see it can be simplified somewhat. We have our sagemaker examples here if you want to take a look https://github.com/MantisAI/sagemaker_examples. They are not documented which I might do at some point this week. We also have a planned blog around inference that we never posted which I now added to the queue so that it can go live.

The ingradients for a sagemaker deployment are

a Dockerfile that contains the api in a serve file https://github.com/MantisAI/sagemaker_examples/blob/main/Dockerfile.custom-sklearn-inference
the api https://github.com/MantisAI/sagemaker_examples/blob/main/src/sklearn_api.py
code to deploy the endpoint
https://github.com/MantisAI/sagemaker_examples/blob/main/scripts/deploy_sagemaker.py
and a script to push the image

It's also worth exploring the even easier option of using the huggingface preconfigured containers because in that case deploy would be as simple as running one script similar to https://github.com/MantisAI/sagemaker_examples/blob/main/scripts/deploy_sklearn_sagemaker.py.

ArneRobben commented 1 year ago

Is the latest model in dvc?

Yes it is!

ArneRobben commented 1 year ago

Thanks for the comments @nsorros. I'll have a look at Mantis' Sagemaker examples as well. Just to clarify - the approach I am following at the moment is the approach I implemented in https://github.com/wellcometrust/sagemaker_test and which on its turn is based on this example from AWS and this TowardsDataScience article

ArneRobben commented 1 year ago

I cleaned up the task.py file a little so this should make more sense now. However, my endpoint fails because Sagemaker can't ping into it.. From the error log:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 2528, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1825, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1823, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "/opt/program/task.py", line 31, in ping
    prediction = predict_tags(
TypeError: predict_tags() missing 1 required positional argument: 'approach'

At the end of last year/beginning of this one, I took out the approach argument from most code in grants_tagger because it triggered a lot of dependencies issues when building a docker container. Oddly enough this seems what is missing now. I pushed the latest version (v0.2.5) to PyPi so I am not sure why it needs this argument..

I'll have to leave it at this unfortunately but I think once figured out what's happening here, Sagemaker should be able to ping in fine and we can test if it can take in a .json for some predictions

nsorros commented 1 year ago

I will close this in favor of using sage to deploy grants tagger, reopen if needed @ArneRobben

wellcometrust / grants_tagger