tnc-ca-geo / animl-api

Backend for https://animl.camera
Other
4 stars 0 forks source link

Update MIRA model interface to support MIRAv2 #93

Closed nathanielrindlaub closed 1 year ago

nathanielrindlaub commented 1 year ago

With the deployment of MIRAv2 to Sagemaker Serverless, we can now invoke the endpoint directly using the Sagemaker SDK (as we now do for Megadetector).

The endpoint expects a multipart form with a payload of:

{
    image: <image encoded as base64 string>
    bbox: <bbox coordinates in relative values in a [ymin, xmin, ymax, xmax] array>
}

See the handler and endpoint testing code in the mirav2_deploy.ipynb to see how the invocation works in the Python (Boto) SDK:

from io import BytesIO
import boto3
from PIL import Image
import json
import base64

endpoint_name = "mirav2"
image_key = "rodent-full-size.jpg"
image_bbox = [0.3312353789806366, 0.22853891551494598, 0.4267701208591461, 0.436643123626709] 

image = boto3.client("s3").get_object(Bucket="animl-sample-images", Key=image_key)['Body'].read()
image_string = base64.b64encode(image).decode('ascii')
payload = { 'image': image_string, 'bbox': image_bbox } # bbox can either be list or string (e.g. "[0,0,1,1]")
payload_encoded = json.dumps(payload).encode('utf-8')
client = boto3.client("runtime.sagemaker")
response = client.invoke_endpoint(
    EndpointName=endpoint_name, ContentType="multipart/form-data", Body=payload_encoded
)
response = json.loads(response["Body"].read())
nathanielrindlaub commented 1 year ago

We'll need to update the MIRA model record in the DB as well, I imagine.

nathanielrindlaub commented 1 year ago

...also might need to add an SSM param for the endpoint name.

nathanielrindlaub commented 1 year ago

Done