qdrant / qdrant-client

Python client for Qdrant vector search engine
https://qdrant.tech
Apache License 2.0
741 stars 118 forks source link

Don't write .lock file when loading from an existing collection #765

Open jonas-nothnagel opened 2 weeks ago

jonas-nothnagel commented 2 weeks ago

I have stored a qdrant collection locally on my computer to access it for testing purposes. I am building a chatbot app in gradio and it requires to load the app.py many times to develop it in iterations.

However, every time I load the collection a .lockfile is automatically created which I have to manually delete each time when loading app again for my application. Otherwise I run into following error:

RuntimeError: Storage folder qdrant/ is already accessed by another instance of Qdrant client. If you require concurrent access, use Qdrant server instead.

The code to load the collection is as follows:

vectors = QdrantVectorStore.from_existing_collection(
    path="qdrant/", 
    collection_name="uganda-documents",

    embedding = HuggingFaceEmbeddings(
        model_kwargs = {'device': device},
        encode_kwargs = {'normalize_embeddings': bool(int(config.get('retriever','NORMALIZE')))},
        model_name=config.get('retriever','MODEL')
    )
)

I would like to either automatically delete the .lock file or to not create it at all in the first place. Is this possible within Qdrant or do I have to build a manual work around?

I know I could simply add something like a os.remove("qdrant/.lock") but it seems not very intuitive.

joein commented 4 days ago

Hi @jonas-nothnagel, sorry for the late response

I am not an expert of gradio, but if it works in a similar way to streamlit, then you would either have a shared object, which is created only once or you can use .close()

If the former approach is possible, I think it would be preferable