cannot pickle '_thread.RLock' object

codebasics / langchain

Tutorial for langchain LLM library

128 stars 223 forks source link

cannot pickle '_thread.RLock' object #4

Open sambantham-p opened 1 year ago

sambantham-p commented 1 year ago

Traceback (most recent call last): File "D:\news agent\venv\Lib\site-packages\streamlit\runtime\scriptrunner\script_runner.py", line 534, in _run_script exec(code, module.dict) File "D:\news agent\main.py", line 31, in final = qa.qaretriever(question,urls) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\news agent\qaretriever.py", line 35, in qaretriever pickle.dump(vector_index, f) TypeError: cannot pickle '_thread.RLock' object

cabdelahad commented 11 months ago

did you solve this ?

HariPrasanth commented 5 months ago

Hi, Were you able to solve this issue?

HariPrasanth commented 5 months ago

The answer is : We will have to use vectorstore methods only to dump into a file and retrieve it instead of using pickle

Just replaced the code

with open(file_path, "wb") as f:
    pickle.dump(vectorstore_openai, f)

With this line vectorstore_openai.save_local("vectorstore")

And

replace this code vectorstore = pickle.load(f) with vectorstore = FAISS.load_local("vectorstore", OpenAIEmbeddings(), allow_dangerous_deserialization=True)