[Bug]: Getting Sqlite 3 Runtime Error while trying to run python 3.11.4 with Chromadb 0.4.4 on Docker container

[Rasika-Deodhar]

What happened?

I am trying to deploy an application with Python 3.11.4 backend which uses Chromadb 0.4.4. However, I am getting an sqlite3 unsupported version error when I try calling it using AzureChatOpenAI endpoint. This happens specifically when I am trying to deploy this on a docker container. It runs fine on local machine.

Code for reproduction after deployment:

I have my code in 3 parts:

1. Creating a AzureChatOpenAI endpoint using langchain.

   from langchain.chat_models import AzureChatOpenAI 
   openai = AzureChatOpenAI()
   embeddings = OpenAIEmbeddings()

2. Storing into a vectorstore using ChromaDb.

It fails on Chroma.from_documents(texts, embeddings).

    texts = 'some texts from a document'
    db = Chroma.from_documents(texts, embeddings) 
    retriever = db.as_retriever(search_type='similarity')

3. Retrieving responses from the AzureChatOpenAI endpoint.

   qa = ConversationalRetrievalChain.from_llm(openai, retriever)

Versions

### Environment on Docker Container Running on Ubuntu - Linux 5.10.16.3-microsoft-standard x86_64 x86_64 x86_64 GNU/Linux and latest Chroma 0.4.4

• acachecontrol==0.3.5
• aiohttp==3.8.5
• aiosignal==1.3.1
• anyio==3.7.1
• async-timeout==4.0.2
• asyncio==3.4.3
• attrs==23.1.0
• backoff==2.2.1
• blinker==1.6.2
• certifi==2019.11.28
• chardet==3.0.4
• charset-normalizer==3.2.0
• chroma-hnswlib==0.7.2
• chromadb==0.4.4
• click==8.1.6
• coloredlogs==15.0.1
• dataclasses-json==0.5.14
• dbus-python==1.2.16
• decorator==5.1.1
• distro-info===0.23ubuntu1
• ecdsa==0.18.0
• fastapi==0.99.1
• Flask==2.3.2
• Flask-Cors==4.0.0
• flatbuffers==23.5.26
• frozenlist==1.4.0
• greenlet==2.0.2
• gunicorn==20.0.4
• h11==0.14.0
• httptools==0.6.0
• humanfriendly==10.0
• idna==2.8
• importlib-resources==6.0.0
• itsdangerous==2.1.2
• Jinja2==3.1.2
• langchain==0.0.251
• langsmith==0.0.18
• MarkupSafe==2.1.3
• marshmallow==3.20.1
• monotonic==1.6
• mpmath==1.3.0
• multidict==6.0.4
• mypy-extensions==1.0.0
• numexpr==2.8.4
• numpy==1.25.2
• okta-jwt-verifier==0.2.4
• onnxruntime==1.15.1
• openai==0.27.8
• openapi-schema-pydantic==1.2.4
• overrides==7.3.1
• packaging==23.1
• posthog==3.0.1
• protobuf==4.23.4
• pulsar-client==3.2.0
• pyasn1==0.5.0
• pydantic==1.10.12
• PyGObject==3.36.0
• pypdf==3.14.0
• PyPDF2==3.0.1
• PyPika==0.48.9
• pysqlite3-binary==0.5.1
• python-apt==2.0.1+ubuntu0.20.4.1
• python-dateutil==2.8.2
• python-dotenv==1.0.0
• python-jose==3.3.0
• PyYAML==6.0.1
• regex==2023.6.3
• requests==2.31.0
• requests-unixsocket==0.2.0
• retry2==0.9.5
• rsa==4.9
• six==1.14.0
• sniffio==1.3.0
• SQLAlchemy==2.0.19
• starlette==0.27.0
• sympy==1.12
• tenacity==8.2.2
• tiktoken==0.4.0
• tokenizers==0.13.3
• tqdm==4.65.0
• typing-inspect==0.9.0
• typing_extensions==4.7.1
• unattended-upgrades==0.1
• urllib3==2.0.4
• uvicorn==0.23.2
• uvloop==0.17.0
• watchfiles==0.19.0
• websockets==11.0.3
• Werkzeug==2.3.6
• yarl==1.9.2

Tried with Chroma 0.4.2 as well - but didn't work.

Error log images:

Other Fixes Tried:

Fixes tried

1. Followed the documentation from the error log : https://docs.trychroma.com/troubleshooting#sqlite. Added installation in docker image.
2. Tried forced installations of sqlite4 and sqlite5 in docker-api file.
3. Added sqlite import in my script similar to what is mentioned here - https://community.render.com/t/your-system-has-an-unsupported-version-of-sqlite3-chroma-requires-sqlite3-3-35-0/13807

Relevant log output

No response

[jeffchuber]

@Rasika-Deodhar is it possible to use python 3.10 - we use python:3.10-slim-bookworm as the base docker image and it works well.

[deshraj]

Running into the same issue. Would be great if the chroma devs can resolve this issue and make it work for other docker images.

[Rasika-Deodhar]

@Rasika-Deodhar is it possible to use python 3.10 - we use python:3.10-slim-bookworm as the base docker image and it works well.

Thanks a lot for you response. I am afraid I cannot use the earlier version of Python, as it may not align with the compatibility requirements of the other langchain libraries we are aiming to incorporate.

[emilesilvis]

Seeing the same issue with base image python:3.11.3.

[emilesilvis]

Using version 0.3.29 of chromadb seems to work (as per https://community.render.com/t/your-system-has-an-unsupported-version-of-sqlite3-chroma-requires-sqlite3-3-35-0/13807/6).

[HammadB]

@emilesilvis Older versions of chroma don't use sqlite and therefore won't have this issue.

@Rasika-Deodhar can you share how you are swapping the pysqlite binary in? It seems like thats not working for you.

[Toysoldier34]

On MacOS using Chroma 0.4.17 and I run into the SQLite 3 version issue on multiple versions of Python and haven't found one that resolves it, including 3.10.10, 3.10.13, 3.11.6.