facebookresearch / Sphere

Web-scale retrieval for knowledge-intensive NLP
Other
555 stars 27 forks source link

Reasonable size index for tests #10

Open isvanilin opened 1 year ago

isvanilin commented 1 year ago

Good day! Appreciate the work carried out with this huge public knowledge corpus! Did you consider creating a smaller version of the index fitting to 5-50Gb? This could facilitate public experiments with the library and maybe quality loss could be mitigated by ranking the initial index by popular entities / issues and filtering them?