falconn wrapper - Githubissues

VarIr / scikit-hubness

A Python package for hubness analysis and high-dimensional data mining

BSD 3-Clause "New" or "Revised" License

44 stars 10 forks source link

falconn wrapper #20

Closed VarIr closed 4 years ago

VarIr commented 5 years ago

Currently, there seems to be no way to create a valid scikit-learn-compatible wrapper for falconn, as its LSHindex cannot be pickled. This is an issue of falconn.

Possible solutions:

Wait for falconn pickle support (however, no ETA)
Switch to different LSH library
Add other ANN tools, be it LSH or not

VarIr commented 5 years ago

The falconn.LSHindex could possibly be memmapped as done in #21.

VarIr commented 5 years ago

There seems to be no adequate way to serialize/memmap the index. falconn development seems to have stalled, so it's unclear, whether this will be possible at all.

VarIr commented 5 years ago

puffinn can now pickle its index, so phasing out falconn is probably the way to go.