Zilliz : Error while initiating model from pymilvus : cannot import name 'GenerationMixin' from 'transformers.generation'

[MukundaDigitar]

SentenceTransformerEmbeddingFunction is erroring out during initializing the model.dense.SentenceTransformerEmbeddingFunction. Code : from pymilvus import model

sentence_transformer_ef = model.dense.SentenceTransformerEmbeddingFunction( model_name='all-MiniLM-L6-v2', # Specify the model name device='cpu' # Specify the device to use, e.g., 'cpu' or 'cuda:0' ) Error : ERROR:root:An error occurred: cannot import name 'GenerationMixin' from 'transformers.generation' (/usr/local/lib/python3.12/site-packages/transformers/generation/init.py)

Note : This SentenceTransformerEmbeddingFunction was working perfectly with Zilliz until recently. Its always erroring out now on init.

[wxywb]

please list your milvus-model, sentence-transformers,transformers's version.

pip list |grep sentence-transformers
pip list |grep transformers
pip list |grep milvus-model

[MukundaDigitar]

hi @wxywb thanks for your reply. Please note that I dont separately have the sentence-transformers installed via my requirements.txt when running my code. I directly use pymilvus install and then use model. example : my requirements.txt has this : pymilvus

my code has this : from pymilvus import MilvusClient, connections, model, utility

class Processor: def init(self, file_id,model_name="all-MiniLM-L6-v2", device="cpu"): self.model_name = model_name self.device = device self.sentence_transformer_ef = model.dense.SentenceTransformerEmbeddingFunction( model_name=self.model_name, device=self.device ) its in this line where its erroring out : self.sentence_transformer_ef = model.dense.SentenceTransformerEmbeddingFunction( model_name=self.model_name, device=self.device

Help in resolution of this issue will be appreciated. Please note that we were able to successfully initiate and call these methods till recently and were getting sentence embeddings.

thanks Mukunda

[wxywb]

I think this is raised due to version issues, I tested this on colab and it works well. https://colab.research.google.com/drive/1jAEuCTs3a7zD0-FT1xUBderFvcJopVEv#scrollTo=ivOfU6OUdst1 I can not reproduce your problem unless I know all the versions in your current enviroment.

[xiaofan-luan]

which version is it? python or something else

[MukundaDigitar]

hi @xiaofan-luan , @wxywb , I have python 3.11.7 64 bit. pymilvus version isnt specified in my requirements.txt . this issue arises when its put in docker and deployed. other than pymilvus (without specifying any particular version for installation in my requirements.txt), there's no other explicit installation that happens simply cos I use the pymilvus objects and Zilliz pymilvus is taking care of installing the dependencies. I have just copied the code presented here in the official docs. no change other than that. ref doc url : https://milvus.io/api-reference/pymilvus/v2.4.x/EmbeddingModels/SentenceTransformerEmbeddingFunction/SentenceTransformerEmbeddingFunction.md As i said earlier, it used to work till lately, from past few days, this issue has started to occur. earlier docker deployments were also successfully running and this code is not changed at all and has been taken from the above mentioned official doc example code for python.

thanks Mukunda

[wxywb]

Can you show me full stacktrace messages and transformers library version? I need them to debug this. and please also try to execute this in your python enviroment(same env you ran pymilvus).

 from transformers.generation import GenerationMixin

[MukundaDigitar]

hi @wxywb I tried adding "from transformers.generation import GenerationMixin". that didnt work. I am also putting the full stack trace below. Please note that when its calling the SentenceTransformerEmbeddingFunction, logging.info("Initializing SentenceTransformerEmbeddingFunction") self.sentence_transformer_ef = dense.SentenceTransformerEmbeddingFunction( model_name=self.model_name, device=self.device ) logging.info("SentenceTransformerEmbeddingFunction initialized") my requirements are : pymilvus milvus_model

it gets into this error : ERROR: THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them. unknown package: Expected sha256 165764f44ef8c61fcdfdfdbe769d687e06374059fbb388b6c89ecb0e28793a6f Got 10ae133e0223861fae36b8888a70a8e23e711084e24ae519628eaaaf398e1967

ERROR:root:An error occurred: install error Command 'pip install -q sentence-transformers' returned non-zero exit status 1. ERROR:root:Environment error: HTTPConnectionPool(host='localhost', port=8089): Max retries exceeded with url: /files?dFileId=file_y6hhzf6747nkmyz (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f856c784230>: Failed to establish a new connection: [Errno 111] Connection refused')) ERROR: Exception in ASGI application Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/milvus_model/utils/dependency_control.py", line 9, in prompt_install subprocess.check_call(cmd, shell=True) File "/usr/local/lib/python3.12/subprocess.py", line 413, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'pip install -q sentence-transformers' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/uvicorn/protocols/http/h11_impl.py", line 406, in run_asgi
result = await app( # type: ignore[func-returns-value] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 70, in call
return await self.app(scope, receive, send) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/fastapi/applications.py", line 1054, in call await super().call(scope, receive, send) File "/usr/local/lib/python3.12/site-packages/starlette/applications.py", line 113, in call await self.middleware_stack(scope, receive, send) File "/usr/local/lib/python3.12/site-packages/starlette/middleware/errors.py", line 187, in call raise exc File "/usr/local/lib/python3.12/site-packages/starlette/middleware/errors.py", line 165, in call await self.app(scope, receive, _send) File "/usr/local/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 62, in call
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) File "/usr/local/lib/python3.12/site-packages/starlette/_exception_handler.py", line 62, in wrapped_app
raise exc File "/usr/local/lib/python3.12/site-packages/starlette/_exception_handler.py", line 51, in wrapped_app
await app(scope, receive, sender) File "/usr/local/lib/python3.12/site-packages/starlette/routing.py", line 715, in call await self.middleware_stack(scope, receive, send) File "/usr/local/lib/python3.12/site-packages/starlette/routing.py", line 735, in app await route.handle(scope, receive, send) File "/usr/local/lib/python3.12/site-packages/starlette/routing.py", line 288, in handle await self.app(scope, receive, send) File "/usr/local/lib/python3.12/site-packages/starlette/routing.py", line 76, in app await wrap_app_handling_exceptions(app, request)(scope, receive, send) File "/usr/local/lib/python3.12/site-packages/starlette/_exception_handler.py", line 62, in wrapped_app
raise exc File "/usr/local/lib/python3.12/site-packages/starlette/_exception_handler.py", line 51, in wrapped_app
await app(scope, receive, sender) File "/usr/local/lib/python3.12/site-packages/starlette/routing.py", line 74, in app await response(scope, receive, send) File "/usr/local/lib/python3.12/site-packages/starlette/responses.py", line 156, in call await self.background() File "/usr/local/lib/python3.12/site-packages/starlette/background.py", line 41, in call await task() File "/usr/local/lib/python3.12/site-packages/starlette/background.py", line 28, in call await run_in_threadpool(self.func, *self.args, *self.kwargs) File "/usr/local/lib/python3.12/site-packages/starlette/concurrency.py", line 39, in run_in_threadpool
return await anyio.to_thread.run_sync(func, args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/anyio/to_thread.py", line 56, in run_sync return await get_async_backend().run_sync_in_worker_thread( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 2177, in run_sync_in_worker_thread return await future ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 859, in run result = context.run(func, args) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/main.py", line 91, in process_file_in_background is_trigger2_complete = process_file( ^^^^^^^^^^^^^ File "/app/main.py", line 54, in process_file processor = ModifiedIPProcessor(file_name, dTenantId, fileId) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/app/embeddings/modified_ip_processor.py", line 35, in init self.sentence_transformer_ef = dense.SentenceTransformerEmbeddingFunction( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/milvus_model/dense/init.py", line 26, in SentenceTransformerEmbeddingFunction return sentence_transformer.SentenceTransformerEmbeddingFunction(args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/milvus_model/utils/lazy_import.py", line 18, in getattr
module = self._load() ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/milvus_model/utils/lazy_import.py", line 12, in _load module = importlib.import_module(self.name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/importlib/init.py", line 90, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 1387, in _gcd_import File "", line 1360, in _find_and_load File "", line 1331, in _find_and_load_unlocked File "", line 935, in _load_unlocked File "", line 995, in exec_module File "", line 488, in _call_with_frames_removed File "/usr/local/lib/python3.12/site-packages/milvus_model/dense/sentence_transformer.py", line 8, in import_sentence_transformers() File "/usr/local/lib/python3.12/site-packages/milvus_model/utils/init.py", line 28, in import_sentence_transformers _check_library("sentence_transformers", package="sentence-transformers") File "/usr/local/lib/python3.12/site-packages/milvus_model/utils/init.py", line 74, in _check_library
prompt_install(package if package else libname) File "/usr/local/lib/python3.12/site-packages/milvus_model/utils/dependency_control.py", line 12, in prompt_install raise ValueError(f"install error {e}") ValueError: install error Command 'pip install -q sentence-transformers' returned non-zero exit status 1.

[wxywb]

It looks like the download of sentence-transformers failed, possibly due to a network connection issue. Can you try installing it manually?

pip install --no-cache-dir sentence-transformers

[MukundaDigitar]

hi @wxywb the issue was happening every time it tried to install sentence-transformers when it was executing the code. i have separtely installed sentence-transformers to mitigate the issue, but please mark this as a bug as other users might also encounter if they havent separately installed sentence-transformers in their requirements.txt and try to execute the zilliz code directly.

[wxywb]

Based on the error stack, it seems that the downloading of the sentence-transformer failed due to network conditions. It's hard to say that it's a bug in the milvus-model.

ERROR: THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them. unknown package: Expected sha256 165764f44ef8c61fcdfdfdbe769d687e06374059fbb388b6c89ecb0e28793a6f Got 10ae133e0223861fae36b8888a70a8e23e711084e24ae519628eaaaf398e1967

ERROR:root:An error occurred: install error Command 'pip install -q sentence-transformers' returned non-zero exit status 1. ERROR:root:Environment error: HTTPConnectionPool(host='localhost', port=8089): Max retries exceeded with url: /files?dFileId=file_y6hhzf6747nkmyz (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f856c784230>: Failed to establish a new connection: [Errno 111] Connection refused'))

[codingjaguar]

I think this is simply due to network condition. Retry with fixing the network (say using huggingface proxy) or manually download the model weights should fix the problem.