search

awslabs / multi-model-server

Multi Model Server is a tool for serving neural net models for inference
Apache License 2.0
995 stars 231 forks source link

Method to know the index of workers? #908

Closed muhyun closed 4 years ago

muhyun commented 4 years ago

To run multiple workers in an instance with multiple GPUs, I need to specify which device to use, but I don't know how to assign device ID (or GPU ID) via mxnet.gpu() without duplication. Is there good way of doing this? For example, I want the first worker use mx.gpu(0), the second worker use mx.gpu(1), and so on.

muhyun commented 4 years ago

This should be handled by sagemaker inference toolkit not by mms.