Easily use and train state of the art late-interaction retrieval methods (ColBERT) in any RAG pipeline. Designed for modularity and ease-of-use, backed by research.
Apache License 2.0
2.45k
stars
173
forks
source link
Index metadata does not change after add_to_index or delete_from_index #192
After I built an index, I thought I was being clever by reading in docid_metadata_map.json to be able to review metadata and select documents I want to work with. However, when adding (add_to_index) or removing (delete_from_index) from the index, it appears that while the operations seem successful, the docid_metadata_map.json and pid_docid_map.json files are not updated.
Is this expected behavior? I'd think we'd want to have those files updated to allow for effective metadata management.
After I built an index, I thought I was being clever by reading in
docid_metadata_map.jsonto be able to review metadata and select documents I want to work with. However, when adding (add_to_index) or removing (delete_from_index) from the index, it appears that while the operations seem successful, thedocid_metadata_map.jsonandpid_docid_map.jsonfiles are not updated.Is this expected behavior? I'd think we'd want to have those files updated to allow for effective metadata management.
Thanks for any insights.