Open axiangcoding opened 5 days ago
Looks like the issue has been mentioned before at https://github.com/chroma-core/chroma/issues/1009#issuecomment-1695083394, and https://github.com/chroma-core/chroma/issues/1152 Could it only be the issue in Windows?
Hey @axiangcoding, you are hitting a Windows-related problem where (not necessarily your case) a Windows admin process (e.g., MS Defender) holds the file for a little while after another process has accessed it.
However, your case might be slightly different as your FastAPI can hold the file, given that your cronjob is a separate process. Before trying to run shutil.rmtree(presist_path)
do you delete the given collection for which you're cleaning up the data (e.g. client.delete_collection("col_name")
?
hi @tazarov , I have try client.delete_collection()
before remove directory, but it only cleaned up the data in the sqlite database and didn't delete any files.
btw, i tried client.reset()
too, didn't delete any files.
@axiangcoding, can you trace what process keeps the lock on the file? As an alternative, have you tried running things into a docker container with a volume instead of directory mount?
@axiangcoding, can you trace what process keeps the lock on the file? As an alternative, have you tried running things into a docker container with a volume instead of directory mount?
Just known that it is the fastapi process that keep the file lock. I'll try the other sugguestions later
@axiangcoding, thanks for confirming. That means that Chroma hasn't released the file yet.
What happened?
I have two programmes, one build by fastapi, we call it
server
, and one for schedule tasks, we call itcronjob
. I'm using chromadb inserver
to create the data through chromadb sdk (wrapper by langchain-chromadb), and i have running thecronjob
to clean the persist directory.But when chromadb finishes executing normally in
server
, and then I delete the persist directory on thecronjob
, error happened.code in
server
is as follow:code in
cronjob
is as follow:This is very strange. It looks like some resources are not being released. Any idea of where should i start to debug it?
Versions
both service install the same version of chromadb
langchain-chroma = "^0.1.1" chromadb = "^0.5.3"
Relevant log output