feat(faiss): supporting soft delete and incremental indexing - Githubissues

jina-ai / executors

internal-only

Apache License 2.0

31 stars 12 forks source link

feat(faiss): supporting soft delete and incremental indexing #159

Closed numb3r3 closed 3 years ago

numb3r3 commented 3 years ago

The preliminary is https://github.com/jina-ai/executors/pull/96

[x] add soft deletion support
- [x] unit test
[x] add update support (low priority)
[x] add snapshot support
- [x] unit test
- [x] integration test
[ ] check performance degradation
- [ ] Train with 1kk and index 1kk results
- [ ] Train with X and index 1kk results (x in [100K, 200K, 500K, 800K])