milvus-io / milvus-sdk-go

Go SDK for Milvus.
Apache License 2.0
347 stars 109 forks source link

[Bug]: Getting sparse vector value returns byte encrypted array #791

Open juanandreas opened 3 months ago

juanandreas commented 3 months ago

Is there an existing issue for this?

Current Behavior

When I run this code to get the content of a sparse float vector:

searchResult.Fields.GetColumn("sparse_vector_column").FieldData().GetVectors().GetSparseFloatVector().GetContents()

I get:

["GwEAAAAAgD82AQAAAACAP0IBAAAAAIA/RAEAAAAAgD+IAgAAAACAP4kCAAAAAIA/IQMAAAAAgD8iAwAAAACAPyMDAADXo1A/","GwEAAAAAgD82AQAAAACAP0IBAAAAAIA/RAEAAAAAgD+IAgAAAACAP4kCAAAAAIA/IQMAAAAAgD8iAwAAAACAPyMDAADXo1A/","GwEAAAAAgD82AQAAAACAP0IBAAAAAIA/RAEAAAAAgD+IAgAAAACAP4kCAAAAAIA/IQMAAAAAgD8iAwAAAACAPyMDAABcj0I/","GwEAAAAAgD82AQAAAACAP0IBAAAAAIA/RAEAAAAAgD+IAgAAAACAP4kCAAAAAIA/IQMAAAAAgD8iAwAAAACAPyMDAAAMAis/","QwEAAAAAgD9EAQAAAACAP2gBAAAAAIA/YAIAAAAAgD97AgAAAACAP4cCAAAAAIA/iQIAAAAAgD8hAwAAAACAPyIDAAAAAIA/IwMAAB1aJD8=","GwEAAAAAgD82AQAAAACAP0IBAAAAAIA/RAEAAAAAgD+IAgAAAACAP4kCAAAAAIA/IQMAAAAAgD8iAwAAAACAPyMDAADjpRs/","QwEAAAAAgD9EAQAAAACAP2sBAAAAAIA/YAIAAAAAgD97AgAAAACAP4cCAAAAAIA/iQIAAAAAgD8hAwAAAACAPyIDAAAAAIA/IwMAAB+FGz8=","GwEAAAAAgD82AQAAAACAP0IBAAAAAIA/RAEAAAAAgD9rAQAAAACAP2ACAAAAAIA/ewIAAAAAgD+HAgAAAACAP4kCAAAAAIA/IQMAAAAAgD8iAwAAAACAPyMDAACamTk/","GwEAAAAAgD82AQAAAACAP0IBAAAAAIA/RAEAAAAAgD9rAQAAAACAP2ACAAAAAIA/ewIAAAAAgD+HAgAAAACAP4kCAAAAAIA/IQMAAAAAgD8iAwAAAACAPyMDAAAMAis/","GwEAAAAAgD82AQAAAACAP0IBAAAAAIA/RAEAAAAAgD+IAgAAAACAP4kCAAAAAIA/IQMAAAAAgD8iAwAAAACAPyMDAADRIts+","GwEAAAAAgD82AQAAAACAP0IBAAAAAIA/RAEAAAAAgD9JAQAAAACAP2sBAAAAAIA/bQEAAAAAgD91AQAAAACAP10CAAAAAIA/YAIAAAAAgD93AgAAAACAP3sCAAAAAIA/hgIAAAAAgD+HAgAAAACAP4kCAAAAAIA/IQMAAAAAgD8iAwAAAACAPyMDAAArhxY/","QwEAAAAAgD9EAQAAAACAP2sBAAAAAIA/YAIAAAAAgD97AgAAAACAP4cCAAAAAIA/iQIAAAAAgD8hAwAAAACAPyIDAAAAAIA/IwMAAC/dpD4=","GwEAAAAAgD82AQAAAACAP0IBAAAAAIA/RAEAAAAAgD9ZAQAAAACAP2sBAAAAAIA/YAIAAAAAgD97AgAAAACAP4cCAAAAAIA/iQIAAAAAgD8hAwAAAACAPyIDAAAAAIA/IwMAAD0Klz4="]

Expected Behavior

This should return me a hashmap of an integer to float32 value mapping.

Steps To Reproduce

No response

Environment

No response

Anything else?

No response

aoiasd commented 3 months ago

FieldData() used to get result proto data,so you will get the source data like []byte

Usually we could get colum data like hits.Fields.GetColumn("pk").GetAsInt64(id) but now we seems lack GetAsSparseVector.

Maybe you can get sparse data by

vector, err := hits.Fields.GetColumn("vector").Get(id)
if err != nil {
    return err
}
sparse,ok:= vector.(entity.SparseEmbedding)

or

vectorColumn := hits.Fields.GetColumn("vector").(*entity.ColumnSparseFloatVector)
sparse, err := vectorColumn.ValueByIdx(idx)
if err != nil {
    return err
}

But now SparseEmbedding only support get position and value by index, not return a map.

Seems we should complete GetSparseVector for Column and add Mapping method for SparseEmbedding.