I added command-line options to Indexer and SearchPerfTest to take .vec file and dimension and index/search precomputed vectors e.g. as produced by the infer_token_vectors_cohere.py. I've also successfully run that tool to create the .vec files (I'll upload these to home.apache.org soon).
I'm currently trying to run a simple A/A benchmark that indexes and searches these vectors (test the end-to-end path of these changes) and if that works well, once we get this merged, I'll turn these on in nightlies ...
I'm not sure what to expect about these Cohere vectors vs the vectors the nightlies now use. How exactly are they different?
I added command-line options to Indexer and SearchPerfTest to take
.vecfile and dimension and index/search precomputed vectors e.g. as produced by theinfer_token_vectors_cohere.py. I've also successfully run that tool to create the.vecfiles (I'll upload these tohome.apache.orgsoon).I'm currently trying to run a simple A/A benchmark that indexes and searches these vectors (test the end-to-end path of these changes) and if that works well, once we get this merged, I'll turn these on in nightlies ...
I'm not sure what to expect about these Cohere vectors vs the vectors the nightlies now use. How exactly are they different?