tensorchord / pgvecto.rs

Scalable, Low-latency and Hybrid-enabled Vector Search in Postgres. Revolutionize Vector Search, not Database.
https://docs.pgvecto.rs/getting-started/overview.html
Apache License 2.0
1.75k stars 71 forks source link

Feature Request: Add Sum Aggregation and Column-Wise Multiplication for Sparse Vectors #475

Open Barabanga opened 5 months ago

Barabanga commented 5 months ago

Starting off, we'd like to express our appreciacion for the project.

Integrating the following operations for sparse vectors would be of great use to our use case:

By incorporating these functionalities, we can streamline computations directly within the database, sparing us the overhead of loading them into our application.

Thank you for considering this request.

VoVAllen commented 5 months ago

Thanks for your interest! May I ask what's your scenario to do the aggregate sum for sparse vectors? What's the query looks like?

VoVAllen commented 5 months ago

And what's Column-Wise Multiplication? Do you mean dot product between sparse vectors?

Barabanga commented 5 months ago

Thanks for your interest! May I ask what's your scenario to do the aggregate sum for sparse vectors? What's the query looks like?

We utilize a sparse vector as a bag-of-words representation for a set of documents. In this representation, each dimension in the sparse vector denotes a term, while the value indicates its frequency within a given document. By using the sum aggregation, we would be able to easily compute some statistics about the collection. For example select sum(term_frequencies) from documents; would result in the total term frequencies for each term.

And what's Column-Wise Multiplication? Do you mean dot product between sparse vectors?

My bad, I meant element-wise multiplication, not column-wise. For example [1, 2, 3] * [1, 2, 3] = [1, 4, 9] (for both dense and sparse vectors)

VoVAllen commented 5 months ago

Understand! PRs on the way