run-llama / llama_index

LlamaIndex is a data framework for your LLM applications
https://docs.llamaindex.ai
MIT License
34.7k stars 4.9k forks source link

[Feature Request]: Delete not yet implemented for Faiss index. #6250

Closed LF112 closed 9 months ago

LF112 commented 1 year ago

Feature Description

When I use the delete_ref_doc index node deletion method provided by GPTVectorStoreIndex, I get an error: "Delete not yet implemented for Faiss index."

When will the function to delete a node from FAISS be implemented? Is there an alternative way to delete FAISS nodes?

https://github.com/jerryjliu/llama_index/blob/main/llama_index/vector_stores/faiss.py#L142

Reason

Llama Index does not currently have a function to delete nodes from FAISS.

Value of Feature

The delete function is a key part of the process.

jon-chuang commented 1 year ago

Actually, faiss index is currently broken with the latest faiss-cpu==1.7.4.

  File "/home/jonch/Desktop/Programming/mlsys/llama_index/llama_index/vector_stores/faiss.py", line 105, in add
    self._faiss_index.add_with_ids(text_embedding_np)
TypeError: handle_Index.<locals>.replacement_add_with_ids() missing 1 required positional argument: 'ids'

It is currently not a dep of llama_index. I believe we should support the latest and also add a working version as an optional dep of llama_index in setup.py at least.

nandhakumarsoliton commented 1 year ago

Facing the same issue. - Requesting for Delete feature in FAISS. I wish to do some modifications in my Embeddings, I am able to get the node_is that needs to be modified with the help of mrtadata from index.ref_doc_info. But loaded_index.delete_ref_doc(node_id) does throw an error stating - image

How to delete the nodes from the index?

dosubot[bot] commented 9 months ago

Hi, @LF112! I'm Dosu, and I'm here to help the LlamaIndex team manage their backlog. I wanted to let you know that we are marking this issue as stale.

From what I understand, you requested a delete function for nodes in the FAISS index used by Llama Index. You were wondering when this function would be implemented and if there is an alternative way to delete nodes in FAISS. There is also a comment from user "jon-chuang" mentioning that the FAISS index is currently broken with the latest version and suggesting to add a working version as an optional dependency. Another user "nandhakumarsoliton" is facing the same issue and requesting a delete feature in FAISS. They also mention an error they encountered when trying to delete nodes.

Before we proceed, could you please confirm if this issue is still relevant to the latest version of the LlamaIndex repository? If it is, please let us know by commenting on this issue. Otherwise, feel free to close the issue yourself or it will be automatically closed in 7 days.

Thank you for your understanding and cooperation. We appreciate your contribution to LlamaIndex!

Best regards, Dosu