search

tensorchord / pgvecto.rs

Scalable, Low-latency and Hybrid-enabled Vector Search in Postgres. Revolutionize Vector Search, not Database.
https://docs.pgvecto.rs/getting-started/overview.html
Apache License 2.0
1.66k stars 62 forks source link

Any plan to speed up pgvector.rs with Intel AMX instruction? #547

Open george-gu-2021 opened 1 month ago

george-gu-2021 commented 1 month ago

I came across the highly creative project which addresses the vector search inside PosgreSQL, one of the most popular open source databases.

Intel Xeon platform supports AMX instruction which can accelerate vector inner-product computation significantly. May I know if we have a plan to speed up pgvevtor.rs with the advanced CPU instruction? @gaocegege

VoVAllen commented 1 month ago

I had some investigation with AMX before. The conclusion I got is that AMX won't provide speed up for dot product. It's possible to provide some speed up but not too much on matrix vector multiplication. Is this right?

We're interested in AMX but doesn't know it a lot. Would be nice if we can have some collaborations here!

george-gu-2021 commented 1 month ago

Regarding the gain from AMX instruction, it may depend on the scenario and usage method.

May I know the vectors are stored in continuous memory in column oriented? If it is, AMX may speed up ~4X for single query in index flat algorithm per our estimation.

VoVAllen commented 1 month ago

The storage layout is flexible for us, and we're also doing some new algorithm on IVF. Therefore we're definitely interested on the acceleration for vector matrix distance computation. Is there any real world AMX code we can learn from?

george-gu-2021 commented 1 month ago

Here is the PR which illustrates the AMX acceleration for FAISS library for your reference. https://github.com/facebookresearch/faiss/pull/3266