search

chroma-core / chroma

the AI-native open-source embedding database
https://www.trychroma.com/
Apache License 2.0
14.77k stars 1.23k forks source link

[Bug]: Your system has an unsupported version of sqlite3. Chroma requires sqlite3 >= 3.35.0 #1985

Open ShaditCuber opened 5 months ago

ShaditCuber commented 5 months ago

Hello, I am using chromadb locally on windows operating system. When I want to deploy to AWS + Docker + ECR , I have an error when running the entire service since chroma db asks me for sqlite3 >= 3.35.0. And I have no control over the docker I use, since the image I get is the following

FROM public.ecr.aws/lambda/python:3.10

All this is set up with a FastApi

Versions

Last Version 0.4.24

Relevant log output

No response

Chrisvd9 commented 5 months ago

I have the same Issue. Please help!

adrian-valente commented 5 months ago

Not in the org, but I saw in chromadb/init.py, lines 49-85 there is a workaround for colab that has this issue, maybe it is sufficient for you to hardcode the IN_COLAB variable in your case? An environment variable-based system could be a more permanent fix.

tazarov commented 5 months ago

@ShaditCuber, I just looked at the AWS lambda python 3.10 image public.ecr.aws/lambda/python:3.10 and it seems that it has very old version of sqlite3 - 3.7.1-ish. The image itself is base of Amazon Linux (centos fork).

One way to go about it is to build your own image off of the official lambda python image that will fix the sqlite3 problems (e.g. upgrade the sqlite3 version, I think I have somewhere a short script how to install sqlite3 from scratch on centos, gotta look for it).

Out of curiosity, why do you install chromadb core package on a lambda? If you plan to load up the data on every lambda call you might be looking at a hefty AWS bill, should you try to scale. Have you looked at chromadb-client library, it is only the thin client that uses HTTP to talk to a stateful Chroma instance and is a better fit for lambda or serverless workloads.

tazarov commented 5 months ago

@Chrisvd9, what is your setup? Is it also lambda?

AndrewTsai0406 commented 5 months ago

This worked for me.

streamlit: Your system has an unsupported version of sqlite3. Chroma requires sqlite3 >= 3.35.0

I simply pasted 


__import__('pysqlite3')
import sys
sys.modules['sqlite3'] = sys.modules.pop('pysqlite3')

at the begining of __init__.py

vsuhas9 commented 2 months ago

Making this changes In chromadb/init.py worked for me.

https://github.com/chroma-core/chroma/compare/main...vsuhas9:chroma:patch-1?diff=unified&w=