bentoml / BentoML

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
https://bentoml.com
Apache License 2.0
7.14k stars 791 forks source link

Conflict with other packages using the same proto file name #1426

Closed withsmilo closed 2 years ago

withsmilo commented 3 years ago

Describe the bug

I found a bug where BentoML conflicts with other packages using the same proto filename as the status.proto used by BentoML. For example, pymilvus 0.4.0 (https://github.com/milvus-io/pymilvus/blob/v0.4.0/grpc-proto/status.proto).

To Reproduce

Please follow up BentoML's helloworld after importing pymilvus.

from milvus import Milvus

from sklearn import svm
from sklearn import datasets
iris = datasets.load_iris()
X, y = iris.data, iris.target
clf = svm.SVC(gamma='scale')
clf.fit(X, y)

from iris_classifier import IrisClassifier
iris_classifier_service = IrisClassifier()
iris_classifier_service.pack('model', clf)
saved_path = iris_classifier_service.save()

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/user/anaconda3/envs/bento_py37/lib/python3.7/site-packages/bentoml/service/__init__.py", line 696, in save
    from bentoml.yatai.client import get_yatai_client
  File "/Users/user/anaconda3/envs/bento_py37/lib/python3.7/site-packages/bentoml/yatai/client/__init__.py", line 21, in <module>
    from bentoml.yatai.client.bento_repository_api import BentoRepositoryAPIClient
  File "/Users/user/anaconda3/envs/bento_py37/lib/python3.7/site-packages/bentoml/yatai/client/bento_repository_api.py", line 31, in <module>
    from bentoml.yatai.proto.repository_pb2 import (
  File "/Users/user/anaconda3/envs/bento_py37/lib/python3.7/site-packages/bentoml/yatai/proto/repository_pb2.py", line 16, in <module>
    import bentoml.yatai.proto.status_pb2 as status__pb2
  File "/Users/user/anaconda3/envs/bento_py37/lib/python3.7/site-packages/bentoml/yatai/proto/status_pb2.py", line 22, in <module>
    serialized_pb=b'\n\x0cstatus.proto\x12\x07\x62\x65ntoml\"\xd1\x03\n\x06Status\x12)\n\x0bstatus_code\x18\x01 \x01(\x0e\x32\x14.bentoml.Status.Code\x12\x15\n\rerror_message\x18\x02 \x01(\t\"\x84\x03\n\x04\x43ode\x12\x06\n\x02OK\x10\x00\x12\r\n\tCANCELLED\x10\x01\x12\x0b\n\x07UNKNOWN\x10\x02\x12\x14\n\x10INVALID_ARGUMENT\x10\x03\x12\x15\n\x11\x44\x45\x41\x44LINE_EXCEEDED\x10\x04\x12\r\n\tNOT_FOUND\x10\x05\x12\x12\n\x0e\x41LREADY_EXISTS\x10\x06\x12\x15\n\x11PERMISSION_DENIED\x10\x07\x12\x13\n\x0fUNAUTHENTICATED\x10\x10\x12\x16\n\x12RESOURCE_EXHAUSTED\x10\x08\x12\x17\n\x13\x46\x41ILED_PRECONDITION\x10\t\x12\x0b\n\x07\x41\x42ORTED\x10\n\x12\x10\n\x0cOUT_OF_RANGE\x10\x0b\x12\x11\n\rUNIMPLEMENTED\x10\x0c\x12\x0c\n\x08INTERNAL\x10\r\x12\x0f\n\x0bUNAVAILABLE\x10\x0e\x12\r\n\tDATA_LOSS\x10\x0f\x12K\nGDO_NOT_USE_RESERVED_FOR_FUTURE_EXPANSION_USE_DEFAULT_IN_SWITCH_INSTEAD_\x10\x14\x62\x06proto3'
  File "/Users/user/anaconda3/envs/bento_py37/lib/python3.7/site-packages/google/protobuf/descriptor.py", line 942, in __new__
    return _message.default_pool.AddSerializedFile(serialized_pb)
TypeError: Couldn't build proto file into descriptor pool!
Invalid proto descriptor for file "status.proto":
  status.proto: A file with this name is already in the pool.

Expected behavior

Screenshots/Logs

Environment:

Additional context

For workaround, we should set this environment variable before importing BentoML and Milvus.

export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION='python'
stale[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

parano commented 2 years ago

BentoML 1.0 no longer depends on proto and we've simplified the communication with Yatai to REST APIs to avoid the dependency on protobuf and gRPC.

That being said, we won't consider supporting this in older 0.13 branch. If you ran into this issue, we'd recommend migrating to BentoML version 1.0.