Does Multi-model-server support Batch Transform?

[yinsong1986]

I am trying to find docs to see how to use Multi-model-server to do Batch Transform (https://docs.aws.amazon.com/sagemaker/latest/dg/batch-transform.html), but cannot find any docs.

Does Multi-model-server support Batch Transform or plan to do so?

Thank you!

[vdantu]

MMS is compatible with batch-transforms. Do you see issues when using MMS with batch-transform?

MMS also provides a pluggable REST endpoint for GET /execution-parameters .

[yinsong1986]

Hi @vdantu Thx for your reply!

When deployed an multi-model-server, I could invoke different model for inference by set the TargetModel parameter in InvokeEndpoint (https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_runtime_InvokeEndpoint.html). It is not clear to me how to select model in Batch Transform (https://docs.aws.amazon.com/sagemaker/latest/dg/batch-transform.html). Or if we do batch transform without specifying any target model, will the batch transform call all the multiple models and output the inference results? Do u have any idea? Thanks!

[yinsong1986]

Pls see the example code at https://github.com/aws-samples/amazon-sagemaker-predict-electricity-demand-with-custom-gluonts-container/blob/main/02_deploy_gluonts_forecast_models_as_multi_model_endpoints.ipynb

[yinsong1986]

The MMS does not seem to support batch transform directly, to perform batch tranform. We need to create models seperately in Sagemaker, and do the batch transform for each model one by one. Below shows an example of how to do batch transoform for one model.