There appear to be duplicate embeddings in the production embedding store

[jkomoros]

Running reindexCardEmbeddings appears to add embeddings even ones that should already be in the store?

If you look at any given point ID and search by similar you'll find a huge number of duplicates with the same card id and version.

[jkomoros]

Yeah, the number of vectors in production is 100k, instead of the expected ~15k, 10x larger than expected, likely due to these extra reembeddings.

[jkomoros]

• [x] Figure out why every time reindexCardEmbeddings is called it's storing a whole new item (did it ever work?)
• [x] Get gulp configure-qdrant to work again
• [x] Re deploy reindex cardEmbeddings in dev and then hit the endpoint and verify it works
• [x] Update private config on macbook
• [x] Blow away the whole dev index and rebuild
• [x] Blow away the whole prod build, then put in th enew cluster and API key in the config, then gulp configure-qdrant and also npm run generate:config and npm run generate:env
• [x] After verifying that worked ^, Blow away the whole prod index and rebuild
• [x] Switch back to free tier of Qdrant once it's under control and embedding index rebuilt

[jkomoros]

My guess is that it's in reindexCardEmbeddings, it's bulk-fetching all of the items, but the cardsInfo is coming back incorreclty. Looks like it gets the content field but not the card_id field?

[jkomoros]

BTW this "lots of duplicates of the same card content and embedding" is likely why the semanticSort in #688 was finding so many non-existent embeddings? Maybe? Because it was fetching a random embedding for that cardID?

[jkomoros]

The bug that has been fixed was leading to a lot of duplicate embeddings being stored, every time reindexCardEmbeddings was run, which was on every deploy.

[jkomoros]

This is now fixed and deployed into production