awslabs / multi-model-server

Multi Model Server is a tool for serving neural net models for inference
Apache License 2.0
984 stars 230 forks source link

Model specific custom python package installation #1004

Open salvadiswar02 opened 1 year ago

salvadiswar02 commented 1 year ago

Hi,

Is it possible to register models into MMS with user specific custom python packages installed during model registration.

Example Scenario: User 1 has model A, with a specific set of python dependencies (ex Tensorflow = 1.8.0) and its already registered in Multi Model server. Now User 2 wants to register model B, which again uses Tensorflow's latest version (2.8.0) into the same Multi Model server.

Question: Now how can user manage different versions of same python package within Multi Model server for different models. Does Multi Model server provide user an option install model specific python package installation to allow seamless model serving.