unum-cloud / usearch

Fast Open-Source Search & Clustering engine × for Vectors & 🔜 Strings × in C++, C, Python, JavaScript, Rust, Java, Objective-C, Swift, C#, GoLang, and Wolfram 🔍
https://unum-cloud.github.io/usearch/
Apache License 2.0
2.28k stars 143 forks source link

Bug: getting irrelevant closest items when using dtype='bf16' (default) instead of dtype='f16' #505

Open alexyalunin opened 1 month ago

alexyalunin commented 1 month ago

Describe the bug

Just updated library from 2.12.0 to 2.15.3 Now I'm getting irrelevant closest items.

I expect for an anchor item to be closest to it by any metric (I use cos), you can see in Expected behavior screenshot. Now after update anchor item is not closest + I get a lot of irrelevant items for it (see the screenshot).

It turns out the difference is in dtype='bf16' which is default in new version. After reverting it back to dtype='f16' I get an expected behabior.

Screenshot 2024-10-14 at 17 03 45

Steps to reproduce

Here is the code I use

from usearch.index import Index
item_count, dimension = train_matrix.shape
index = Index(
    ndim=dimension, # Define the number of dimensions in input vectors
    metric='cos', # Choose 'l2sq', 'haversine' or other metric, default = 'ip'
    dtype='bf16', # Quantize to 'f16' or 'i8' if needed, default = 'f32'
    # connectivity=16, # Optional: Limit number of neighbors per graph node
    # expansion_add=128, # Optional: Control the recall of indexing
    # expansion_search=64, # Optional: Control the quality of the search
    # multi=False, # Optional: Allow multiple vectors per key, default = False
)

_ = index.add(list(range(item_count)), train_matrix)

k = 100
res = index.search(train_matrix, count=k+1)  # +1 because same product is closest
train_similars = train_ids[res.keys]

Expected behavior

using dtype='f16' Screenshot 2024-10-14 at 17 05 53

USearch version

2.15.3

Operating System

Ubuntu 20.04.6 LTS

Hardware architecture

x86

Which interface are you using?

Python bindings

Contact Details

ale.yalunin@gmail.com

Are you open to being tagged as a contributor?

Is there an existing issue for this?

Code of Conduct

ashvardanian commented 1 month ago

Hi @alexyalunin! Did you only change the dtype in the constructor or somewhere else?

ashvardanian commented 1 month ago

Can you please log the type of the input matrix and the hardware_capabilities of the index?

alexyalunin commented 1 month ago

train_matrix.dtype dtype('float32')

Idk how to log hardware_capabilities, here is index

usearch.Index

alexyalunin commented 1 month ago

I didn't change dtype of index by myself. In the example I changed dtype to bf16 to reproduce the problem

alexyalunin commented 1 month ago

I have just rebuilt the index 5 times in a loop and look at 5 anchor items (i.e. 25 examples).

It seems like even with f16 the problem still exists, it is just less often than with bf16. It also seems like with the old version (2.12.0) the problem doesn't exist, so I guess I will just stay with an old version.

ashvardanian commented 1 month ago

Interesting 🤔 Thank you, @alexyalunin! I will look into it!

ashvardanian commented 1 month ago

Found the issue in the underlying SimSIMD library. Investigating. Hope to merge within 24h.