search

SeldonIO / MLServer

An inference server for your machine learning models, including support for multiple frameworks, multi-model serving and more
https://mlserver.readthedocs.io/en/latest/
Apache License 2.0
659 stars 179 forks source link

Can not import tritonclient #1831

Closed haiminh2001 closed 6 days ago

haiminh2001 commented 1 week ago

Context: I am trying to create a custom model and I have a need to make an inference request to a Triton Inference Server. Problem: As soon as I import tritonclient, the mlserver cannot start. It logs "shutdown of default inference pool complete" because of loading duplicate symbol. Expected behaviour: I can import tritonclient and use it properly.

haiminh2001 commented 6 days ago

I fixed it myself by replacing the default symbol database in model_config_pb2.py and service_pb2.py files with a new database. The default database is stored in the google protobuf module, which is shared among libraries like tritonclient and seldon mlserver that caused conflicts. This is only a walkaround and I barely have experiences with grpc, if anyone has a better solution I would appreciate very much.