Sagemaker serverless deployment with torchserve

This sets up a local deployment for testing with a torchserve container and a sagemaker serverless deployment based for a custom torchserve container that works with sagemaker.

The serverless endpoint is currently running here. It can be accessed with

client = boto3.client("runtime.sagemaker")
response = client.invoke_endpoint(
    EndpointName=endpoint_name, ContentType="application/x-image", Body=payload
)

from the notebook /home/rave/animl/animl-ml/api/megadetectorv5/sagemaker-serverless-endpoint-with-torchserve/notebook/mdv5_deploy.ipynb

Files/Folders to pay attention to for review include notebooks/torchscript-yolov5.ipynb and

├── megadetectorv5 # top level includes scripts for local deployment and packaging the .mar model archive for upload to s3
│   ├── docker_mdv5.sh
│   ├── index_to_name.json
│   ├── mdv5_handler.py
│   ├── README.md
│   ├── sagemaker-serverless-endpoint-with-torchserve  # the deployment that depends on the .mar on s3 in animl-model-zoo
│   │   ├── deployment
│   │   │   ├── config.properties
│   │   │   ├── dockerd-entrypoint.sh
│   │   │   ├── handler.py
│   │   │   └── model.py
│   │   ├── Dockerfile
│   │   ├── LICENSE
│   │   ├── notebook
│   │   │   └── mdv5_deploy.ipynb
│   │   └── README.md

the yolov5 folder can be ignored, it was copied to provide easy access to the export.py submodule since the pypi package for it is old and unofficial.

tnc-ca-geo / animl-ml

Sagemaker serverless deployment with torchserve #80