Closed 5antelope closed 7 years ago
is this going to work? i think you need to check that it's modulo the size of a vector, not just the number of dimensions
also note that this can have false negatives
Thanks @erikbern
I added a unit test for it -- ANNIndexTest#testLoadFile: throw an exception when index dimension does not match with dimension in ANNIndex() constructor.
How about we check (file_size % NODE_SIZE) == 0?
I know that it will still introduce false negatives, but ideally the only way to fix that would be adding metadata in the header? The sanity checks here would help to reduce the chances of mistake :-)
Sorry, I misread the comment.
Add a check in ANNIndex#getNearest() to make sure the vector passed in should always be same size as DIMENSION;
Also, I think the check in load(), although introduces some false positive, would be nice to have as well?
Bump -- can I have a quick review on the change? Thanks! :-)
Hi @erikbern, does the change look reasonable to you? Thanks!
looks like you need to rebase!
also i defer to @a1k0n to determine correctness
Hi @erikbern @a1k0n -- I resolved the conflicts and adjust logic based on 2GB change:
it looks for two cases:
Let me know if the change makes sense to you.
Thanks!
@a1k0n mind taking a look while you are free? Thanks! 😄
@erikbern ready to land 😎
Problem: Try to fix #7
Solution: When we call load() at in constructor, if (file_size % dimension) != 0, it indicates there is a mismatch between index file and passed in dimension