search

JohnLetnev / semanticvectors

Automatically exported from code.google.com/p/semanticvectors
Other
0 stars 0 forks source link

Binary BuildIndex fails with dimension not a multiple of 64. Dimension is basically not well managed at the moment. #47

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
Bumping dimension up to a multiple of 64 inside the vectors package leaves the 
dimension of vectors themselves inconsistent with Flags.dimension. I guess for 
now we should probably just alter the flag value, but I'm getting less and less 
happy with this. Comments and suggestions welcome.

****
java pitt.search.semanticvectors.BuildIndex -dimension 200 -seedlength 100 
-vectortype binary  index/

...

Jul 25, 2011 10:31:08 AM pitt.search.semanticvectors.vectors.BinaryVector <init>
WARNING: Dimensionality set to: 256. This is   an implementation detail to 
facilitate permutation in a manner   consistent with other (non-binary) 
implementations
Exception in thread "main" java.lang.IllegalArgumentException: Trying to add 
vector of dimension 256 to VectorStore of dimension 200
    at pitt.search.semanticvectors.VectorStoreRAM.putVector(VectorStoreRAM.java:85)
    at pitt.search.semanticvectors.DocVectors.initializeDocVectors(DocVectors.java:149)
    at pitt.search.semanticvectors.DocVectors.<init>(DocVectors.java:80)
    at pitt.search.semanticvectors.BuildIndex.main(BuildIndex.java:161)

Original issue reported on code.google.com by widd...@google.com on 25 Jul 2011 at 2:34

GoogleCodeExporter commented 9 years ago 
We've fixed this by only making binary vectors with dimension a multiple of 64, 
and seedlength to be half that. This is done in Flags.makeFlagsCompatible().

I don't love this design, it's part of addressing the problem of configuring 
global variables and their dependencies. But it's done and tested.

Original comment by widd...@google.com on 26 Aug 2011 at 6:46