search

aws / sagemaker-tensorflow-serving-container

A TensorFlow Serving solution for use in SageMaker. This repo is now deprecated.
Apache License 2.0
174 stars 101 forks source link

multi gpu support #227

Closed waytrue17 closed 1 year ago

waytrue17 commented 1 year ago

Description of changes: Enabling multi-gpu support

Tested on a 8 gpu instance with tf_serving_instance 4 and 9. Here is the log for 9 workers:

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A     23000      C   tensorflow_model_server          2275MiB |
|    0   N/A  N/A     25628      C   tensorflow_model_server          2275MiB |
|    1   N/A  N/A     23005      C   tensorflow_model_server          2275MiB |
|    2   N/A  N/A     23269      C   tensorflow_model_server          2275MiB |
|    3   N/A  N/A     23534      C   tensorflow_model_server          2275MiB |
|    4   N/A  N/A     23864      C   tensorflow_model_server          2275MiB |
|    5   N/A  N/A     24466      C   tensorflow_model_server          2275MiB |
|    6   N/A  N/A     25011      C   tensorflow_model_server          2275MiB |
|    7   N/A  N/A     25357      C   tensorflow_model_server          2275MiB |
+-----------------------------------------------------------------------------+

2 workers were assigned to gpu0, which is as expected

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

ashishgupta023 commented 1 year ago

LGTM

sagemaker-bot commented 1 year ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 1 year ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 1 year ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 1 year ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 1 year ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 1 year ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 1 year ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 1 year ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 1 year ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 1 year ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository