search

pixelogik / NearPy

Python framework for fast (approximated) nearest neighbour search in large, high-dimensional data sets using different locality-sensitive hashes.
MIT License
759 stars 152 forks source link

Indexing is too slow. #3

Closed pixelogik closed 9 years ago

pixelogik commented 11 years ago

Indexing is actually NOT slow. What's slow is:

  1. Turning vectors into sparse vectors for compact redis storage
  2. Serializing the vectors into JSON for redis storage
pixelogik commented 9 years ago

This was related to the old "hashsaving" branch. The new support for sparse vectors was developed independent of that branch (which will be deleted now).

When using sparse vectors as supported by scipy storing them / turning them into JSON representations should not be slow anymore.