milvus-io / milvus-sdk-java

Java SDK for Milvus.
https://milvus.io
Apache License 2.0
380 stars 153 forks source link

Not able to insert vectors with FLOAT16 type. #998

Closed daxesh020500 closed 1 month ago

daxesh020500 commented 1 month ago

In my collection, I have a vector field type as Float16Vector with 512 dimensions.

In order to cover Float32 to Float16, I am using Java's Float class. https://docs.oracle.com/en%2Fjava%2Fjavase%2F22%2Fdocs%2Fapi%2F%2F/java.base/java/lang/Float.html#floatToFloat16(float)

And convert each float value to float16, put those two bytes into ByteBuffer. total size is 512*2=1024 as per required. However, when I try to insert the document with 1.1f value, its being converted to 1596.

Am I missing something here? Returns the floating-point binary16 value, encoded in a { short}, Seems because of this, Any other workaround to insert float16 vector into mivlus using java SDK? Mivlus v:2.4.4 Java SDK: 2.4.1

using upsert method of MilvusClient, should this issue be fixed once I move to MilvusClientV2?

cqy123456 commented 1 month ago

Can you give some sample?@daxesh020500

cqy123456 commented 1 month ago

assign @cqy123456

daxesh020500 commented 1 month ago

I had updated the milvus sdk to 2.4.2 and seen one utility class Float16Utils.

Now using the class methods to convert float32 to fp16 and then into a bytebuffer. this worked for my case.