JuliaNeighbors / HNSW.jl

Approximate Nearest Neighbor Searches using the HNSW algorithm
MIT License
21 stars 6 forks source link

Error after building the graph #27

Open MNLubov opened 2 years ago

MNLubov commented 2 years ago

I am trying to use HNSW for the search and have the following problem. After building the graph, I got the folloiwng error: ArgumentError: invalid index: -0.017805025f0 of type Float32 I got this error here: @inline distance(hnsw, i, j) = @inbounds evaluate(hnsw.metric, hnsw.data[i], hnsw.data[j]) How could this happen, if graph is build without problems?

JonasIsensee commented 2 years ago

Hi @MNLubov, can you give me a code example?

MNLubov commented 2 years ago

@JonasIsensee I found what the problem is following. In HNSW distance for knn-search are calculated only for vectors, while for the graph building type of the data is not specified.

@inline distance(hnsw, i, j) = @inbounds evaluate(hnsw.metric, hnsw.data[i], hnsw.data[j])
@inline distance(hnsw, i, q::AbstractVector) = @inbounds evaluate(hnsw.metric, hnsw.data[i], q)
@inline distance(hnsw, q::AbstractVector, j) = @inbounds evaluate(hnsw.metric, hnsw.data[j], q)

I used distance for matrices, not vectors. My distance measure is based on the product of two matrices. Is it possible to extend allowed types for the distance calculation in HNSW? I personally need Float32 matrices:

@inline distance(hnsw, q::Matrix{Float32}, j) = @inbounds evaluate(hnsw.metric, q, hnsw.data[j])
@inline distance(hnsw, i, q::Matrix{Float32}) = @inbounds evaluate(hnsw.metric, q, hnsw.data[i])

Also such addition leads to a change for length assertion:

@assert length(q)==length(hnsw.data[1])

I strongly rely on your package and it would be wonderful if it could be used for any type of data with scalar distance. Moreover change of data type and extension of possible scalar distances doesn't change logic of hnsw search.