search

aws / sagemaker-tensorflow-serving-container

A TensorFlow Serving solution for use in SageMaker. This repo is now deprecated.
Apache License 2.0
174 stars 101 forks source link

feature: expose tunable parameters to support multiple tfs #190

Closed jinpengqi closed 3 years ago

jinpengqi commented 3 years ago

Description of changes:

Implement new container environment which could expose additional parameters to help user optimize for throughput and fine tune container performance.

The feature that multiple of tensorflow model servers could handle parallel requests coming from different gunicorn workers could be supported in this release. And throughput will be improved based on different parameters tuned. Specifically, tunable parameters likeSAGEMAKER_GUNICORN_WORKERS, SAGEMAKER_TFS_INSTANCE_COUNT, SAGEMAKER_TFS_INTER_OP_PARALLELISM, SAGEMAKER_TFS_INTRA_OP_PARALLELISM and etc.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

sagemaker-bot commented 3 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository