When using the inference pipeline, the retriever corpus needs to be rebuilt every time making debugging very time consuming. Instead, there should be an option to use a (precomputed) set of embeddings.
I will gladly create a PR if you find these changes appropriate.
Then, the program can copute a hash of the corpus.tsv and look in the corpus_cache directory to find a file and load the embeddings from there if an appropriate file exists. Otherwise, compute the embeddings anyway and store them.
It is appropiate to load the corpus.tsv into memory in the first place, because we also need access to the plain text, meaning no real "excess work" is being done when loading it and looking up the cache file.
I got this to work by storing just the corpus_embedding as a .tf file:
When using the inference pipeline, the retriever corpus needs to be rebuilt every time making debugging very time consuming. Instead, there should be an option to use a (precomputed) set of embeddings.
I will gladly create a PR if you find these changes appropriate.
Usage might look something like this:
Then, the program can copute a hash of the
corpus.tsvand look in the corpus_cache directory to find a file and load the embeddings from there if an appropriate file exists. Otherwise, compute the embeddings anyway and store them.It is appropiate to load the
corpus.tsvinto memory in the first place, because we also need access to the plain text, meaning no real "excess work" is being done when loading it and looking up the cache file.I got this to work by storing just the corpus_embedding as a
.tffile: