search

jelmerk / hnswlib

Java library for approximate nearest neighbors search using Hierarchical Navigable Small World graphs
Apache License 2.0
260 stars 56 forks source link

Not working in webapp #21

Closed kwalcock closed 4 years ago

kwalcock commented 4 years ago

I have a Scala application which uses an index created by hnswlib and it works great. When I call the exact same code from a play webapp, the deserialization of the TId fails for some reason

Caused by: java.lang.IllegalArgumentException: Could not read input file.
Caused by: java.lang.ClassNotFoundException: <Tid>

I can deserialize this type from elsewhere in the application, even in a webapp, just not during the HnswIndex.load in the same webapp.

Any ideas? Sometimes it is a ClassLoader problem and I vaguely recall seeing an implementation in the code of this repo or one nearby.

kwalcock commented 4 years ago

There is evidence that the problem is with the play framework class loader (https://groups.google.com/forum/#!topic/play-framework/CzLGIYnP8To). It might be helpful if a different class loader could be passed in as an argument.

jelmerk commented 4 years ago

Hi @kwalcock

I made it possible to define your own classloader when loading the index and released the lib as 0.0.46

Can you check if this solves your problem if you use it with the classloader from play.Environment ?

kwalcock commented 4 years ago

Thank you! I will try it out ASAP. A play app can be run in (at least) two ways from within sbt: with run and runProd. With the old code, neither worked. sbt can also be used to stage the app for use outside of sbt. That one uses the normal classloader and did work with your library and it was going to be the workaround. With your changes I hope to be able to run code directly from sbt (or IntelliJ). It will be very helpful.

kwalcock commented 4 years ago

Hartelijk dank. Het blijkt te werken. The default value of Thread.currentThread.getContextClassLoader seems to suffice, so all I needed to do was change from load to loadFromFile.