search

milvus-io / milvus

A cloud-native vector database, storage for next generation AI applications
https://milvus.io
Apache License 2.0
29.71k stars 2.85k forks source link

[Feature]: Ask for Bfloat16 data type insert. #36468

Open mellonyou opened 1 week ago

mellonyou commented 1 week ago

Is there an existing issue for this?

Is your feature request related to a problem? Please describe.

Have added bfloat16 support in knowhere, and query performance have a significate improve. But, directly writing bf16 data type, which need to be converted to bytes, it will results in poor performance.

Describe the solution you'd like.

Can write bfloat16 data type directly without type convert.

Describe an alternate solution.

No response

Anything else? (Additional Context)

No response

xiaofan-luan commented 1 week ago

which SDk are u using? Many of the language don't have Bf16 as datatype, that's why we convert it to binary

mellonyou commented 1 week ago

I test it with python script, and I want to use milvus to receive bf16 data from langchain.

xiaofan-luan commented 1 week ago

I test it with python script, and I want to use milvus to receive bf16 data from langchain.

Did langchain support BF16 embedding already?

zc277584121 commented 1 week ago

As far as I know, langchain itself does not involve data type processing, but generally some deep learning frameworks such as tf, pytorch can generate bf16. @mellonyou Can you please explain your scenario in detail? Are you trying to use langchain_milvus to do this?

congqixia commented 1 week ago

Maybe the need is to integrate langchain output direct into milvus collection with BFloat16 field, which means the input vector is in float32 and let Milvus server do the data type conversion internally

mellonyou commented 1 week ago

@zc277584121 Yes, I am using langchain_milvus to interact with milvus. Actually, I want to integrate milvus into RAG, for dataprep&retrievers. I am trying to setup a bf16 pipeline, the output from embedding is bf16, it need milvus can support bf16 insert&query.

zc277584121 commented 6 days ago

@mellonyou Thank you for your feedback. We will verify the feasibility and try to add some features to meet this requirement in later versions of langchain_milvus.