search

milvus-io / milvus

A cloud-native vector database, storage for next generation AI applications
https://milvus.io
Apache License 2.0
30.68k stars 2.93k forks source link

[Feature]: dose milvus support fp16 or int8 gemm on gpu? #28449

Open HaoLiuHust opened 1 year ago

HaoLiuHust commented 1 year ago

Is there an existing issue for this?

Is your feature request related to a problem? Please describe.

I wonder if milvus support fp16 or int8 matrix multiply on gpu, since fp16 and int8 is much faster than fp32

Describe the solution you'd like.

No response

Describe an alternate solution.

No response

Anything else? (Additional Context)

No response

jiaoew1991 commented 1 year ago

Hi @HaoLiuHust There are several questions that need clarification. The usage scenario of Milvus, a vector database like this, is for ANN search rather than matrix multiplication. In addition, Milvus currently provides two types of vectors: binary and fp32. It also supports CPU-based and GPU-based indexing. Support for fp16 type vectors is under development.

HaoLiuHust commented 1 year ago

Hi @HaoLiuHust There are several questions that need clarification. The usage scenario of Milvus, a vector database like this, is for ANN search rather than matrix multiplication. In addition, Milvus currently provides two types of vectors: binary and fp32. It also supports CPU-based and GPU-based indexing. Support for fp16 type vectors is under development.

I am using it for face recognition, so I prefer to use it with brute force search since ANN may have unacceptable mistake