[Bug]: Error chroma with protobuf

[crissvilleda]

What happened?

Chroma is raising this error "" TypeError: Descriptors cannot be created directly. If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0. If you cannot immediately regenerate your protos, some other possible workarounds are:

1. Downgrade the protobuf package to 3.20.x or lower.
2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).""

Versions

Chroma V.0.5.0 -> V0.5.5, Python 3.10.1

Relevant log output

from langchain_chroma.vectorstores import Chroma
[2024-07-24T17:58:14.972Z]   File "/home/criss/coders/IL-1261A-Functions-chat-Dr-Huang/.venv/lib/python3.10/site-packages/langchain_chroma/vectorstores.py", line 23, in <module>
[2024-07-24T17:58:14.972Z]     import chromadb
[2024-07-24T17:58:14.972Z]   File "/home/criss/coders/IL-1261A-Functions-chat-Dr-Huang/.venv/lib/python3.10/site-packages/chromadb/__init__.py", line 5, in <module>
[2024-07-24T17:58:14.972Z]     from chromadb.auth.token_authn import TokenTransportHeader
[2024-07-24T17:58:14.972Z]   File "/home/criss/coders/IL-1261A-Functions-chat-Dr-Huang/.venv/lib/python3.10/site-packages/chromadb/auth/token_authn/__init__.py", line 20, in <module>
[2024-07-24T17:58:14.972Z]     from chromadb.telemetry.opentelemetry import (
[2024-07-24T17:58:14.972Z]   File "/home/criss/coders/IL-1261A-Functions-chat-Dr-Huang/.venv/lib/python3.10/site-packages/chromadb/telemetry/opentelemetry/__init__.py", line 12, in <module>
[2024-07-24T17:58:14.972Z]     from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
[2024-07-24T17:58:14.972Z]   File "/home/criss/coders/IL-1261A-Functions-chat-Dr-Huang/.venv/lib/python3.10/site-packages/opentelemetry/exporter/otlp/proto/grpc/trace_exporter/__init__.py", line 22, in <module>
[2024-07-24T17:58:14.972Z]     from opentelemetry.exporter.otlp.proto.grpc.exporter import (
[2024-07-24T17:58:14.972Z]   File "/home/criss/coders/IL-1261A-Functions-chat-Dr-Huang/.venv/lib/python3.10/site-packages/opentelemetry/exporter/otlp/proto/grpc/exporter.py", line 39, in <module>
[2024-07-24T17:58:14.972Z]     from opentelemetry.proto.common.v1.common_pb2 import (
[2024-07-24T17:58:14.972Z]   File "/home/criss/coders/IL-1261A-Functions-chat-Dr-Huang/.venv/lib/python3.10/site-packages/opentelemetry/proto/common/v1/common_pb2.py", line 36, in <module>
[2024-07-24T17:58:14.972Z]     _descriptor.FieldDescriptor(
[2024-07-24T17:58:14.972Z]   File "/usr/lib/azure-functions-core-tools-4/workers/python/3.10/LINUX/X64/google/protobuf/descriptor.py", line 561, in __new__
[2024-07-24T17:58:14.972Z]     _message.Message._CheckCalledFromGeneratedFile(

[irowbin]

I am also facing the same issue.

OS: Windows 11
python: 3.11, 3.12 (both shows the same issues)
chroma: 0.5.5, 0.5.0 (same issue)

[tazarov]

@crissvilleda thanks for reporting this:

This is probably a protobuf compatibility issue on the OTEL lib. In our case we don't have an upper bound for the libs:

  'opentelemetry-exporter-otlp-proto-grpc>=1.2.0',

Which may lead to such kind of issues.

May I suggest you downgrade your OTEL grpc lib:

pip install opentelemetry-exporter-otlp-proto-grpc==1.25.0

Optionally also downgrade the other deps:

opentelemetry-sdk==1.25.0
opentelemetry-api==1.25.0

EDIT: I stand corrected. While transitive error indeed originates in the opentelemetry-proto. The root is likely in the protobuf library and possibly the underlying libprotoc

[maxupp]

Also facing this, pretty big issue, as some of my other packages require a newer protobuf version.

[crissvilleda]

I solved it as suggested by @tazarov

[Devansh3712]

Facing the same issue while deploying an app on streamlit community cloud

[maxupp]

We really need a fix, or at least a workaround for this. I am running a service that addresses bot Weaviate (requiring protobuf>5.0.0) and Chroma, which wants <3.20.0.

@tazarov Can you at least make opentelemetry an optional requirement? Since it is an opt-in feature anyway.