Closed andriichumak closed 1 month ago
Hi @andriichumak, yup, it seems like we have to figure out the right way to promote types in this very common scenario for embeddings. Here's the sequence of steps leading to this issue:
sentence-transformers
in Python returns a list of floats, where each item is guaranteed to be of length 384.ARRAY
of length 384embedded
list of floats from Python, which when passed to Kùzu, is cast to DOUBLE[]
, which is a LIST
of doublesarray_cosine_similarity
function works only on ARRAY
types, and in this case, the embedding is being compared against itself, and both are of type LIST
.Note that in Kùzu, ARRAY
is a special case of LIST
- the only difference between a LIST
and an ARRAY
in Kùzu from a user perspective is that the ARRAY
has a fixed length that's known beforehand. When the Python list of floats is passed to Kùzu, it's cast to a LIST
(which is the correct behaviour for reasons of generality) because Python lists are dynamic in nature and their lengths cannot be assumed to be always fixed.
The workaround here is to perform explicit casting of the embedded
variable to the type DOUBLE[384]
, which transforms the LIST
to an ARRAY
, and then it works:
# Slightly rephrase the MATCH query
response = conn.execute(
"""
MATCH (d:MyNode)
WITH d, CAST($emb, "DOUBLE[384]") AS emb
RETURN d.id, array_cosine_similarity(d.embedding, emb)
""", {"emb": embedded}
)
Result:
┌──────┬─────────────────────────────────┐
│ d.id ┆ ARRAY_COSINE_SIMILARITY(d.embe… │
│ --- ┆ --- │
│ str ┆ f64 │
╞══════╪═════════════════════════════════╡
│ test ┆ 1.0 │
└──────┴─────────────────────────────────┘
I think this pattern of usage for embeddings is incredibly common though, and it's not ideal that the user has to perform explicit casting in this manner. It's also a little hard to remember the syntax of explicit casting for new users to Kùzu. Maybe we could make some better assumptions about the fact that users will bring in embeddings from Python libraries like sentence-transformers
, which are guaranteed to return a fixed-length list for an embedding. So it can be considered "safe" for us to promote the LIST
type to ARRAY
inside the array_cosine_similarity
function?
@andyfengHKU, I think we need to put a bit more thought into this, as I faced a similar issue (as no doubt others will) in #3481.
Hey @prrao87. Thanks a lot for a quick feedback. The suggested solution works.
One thing I noticed is that if I define the column type as LIST
instead of fixed length array (i.e. DOUBLE[]
instead of DOUBLE[384]
), it still fails with the same error. Looks like the issue is not that the final execution argument is a list, but rather that it's considered to have the type ANY
for some reason.
UPD: OK, I missed the part that array similarity function does not work on lists. Still, the error message is suspicious, I'd assume both ARRAY
and LIST
should be fine, and the issue is that the typing is lost somewhere along the way.
Yup, fully agree. There's something regarding the behaviour we need to change internally to make this easier, because the way most users bring in embeddings into Kùzu is from numpy/python. @andyfengHKU will have some ideas on this. Thanks for reporting!
Update:
I'd assume both ARRAY and LIST should be fine, and the issue is that the typing is lost somewhere along the way.
Well, similarity calculation only works on same-size lists, so at least one of the two lists being compared must be an array. However, the fact that it assumes the internals of the LIST
are of type ANY
is way too broad (to capture all possibilities), so we need to think about how to cast types internally for this use case without breaking other things.
Hi team. I'm trying to use Kuzu for semantic search using
array_cosine_similarity
and can't get it working. This looks like a bug to me.Here is the minimal repro:
Kuzu 0.4.2 Python 3.11.9