search

aws / sagemaker-tensorflow-serving-container

A TensorFlow Serving solution for use in SageMaker. This repo is now deprecated.
Apache License 2.0
174 stars 101 forks source link

fix: Use default model name when model name is None #214

Open saimidu opened 2 years ago

saimidu commented 2 years ago

Issue #, if available:

Description of changes: The model_name parameter when making a non-MME inference request, in the presence of a user's inference script, is treated as None by the python service for SM inference. This means that the context passed to the inference handler does not have a valid model_name attribute.

Therefore, GRPC-based inference requests for a model cannot be programmatically configured, and must instead always be hardcoded in the user's inference script. This requirement makes it harder to compile a model, or to use it with SageMaker Inference Recommender to produce a Neo-compilation-compatible model.

This PR makes a change to tfs_utils.py to use the default_model_name, obtained from the TFS_DEFAULT_MODEL_NAME environment variable, as the final option in case the input model_name is None and the request attributes do not contain the model name.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository