Does multiple queries makes sense in a vector space?

[DavidGOrtega]

👋

Interesting work! However I do not fully understand the need of generating multiple queries doing a vector search. Once the query is transformed into a vector the search is latent space centric so it will retrieve documents close to our vector. The rewritten queries will be semantically similar and I would not expect much benefit from this. The downside is the cost effectiveness having to multiply the number of queries and vector calculation for each query.

However, a completely different scenario would be generating the different queries in solely text based search DBs where having the same query reformulated with even different words that could retrieve semantically similar documents improving the results.

Have you made some comparison tests?

WDYT?

[Raudaschl]

Hi David,

Your question delves into the core of our methodological approach, and I appreciate the opportunity to clarify the rationale behind generating multiple queries in a vector search context :-)

The primary reason for employing multiple queries, despite the apparent semantic similarity post-vector transformation, is twofold.

First, it aims to capture the nuanced facets of complex queries that a single vector representation might not fully encapsulate. While it’s true that transforming a query into a vector places it in a latent space where documents of close proximity are retrieved, this process might not always grasp the multi-dimensional aspects of certain queries. By generating variations, we aim to explore these latent spaces more thoroughly, ensuring a broader yet still relevant coverage of potential results.

Second, and crucially, generating multiple similar queries and observing the consistency in results serves to build confidence in the robustness and reliability of the search outcomes. When similar but subtly different queries yield the same results, it underscores the search algorithm’s capability to successfully cover multiple perspectives and interpretations of a query, enhancing user trust in the system’s comprehensiveness and accuracy.

The concern regarding cost-effectiveness is certainly valid. However, our approach has been to optimise the vector calculation (small language model) and query generation processes (ChatGPT 3.5) to mitigate these concerns. Preliminary results have indicated that the slight increase in cost is offset by the significant improvement in the relevance and breadth of retrieved documents, especially in highly specialised or nuanced fields.

To your point about text-based search databases, you are correct in noting the distinct advantages of query variation in such contexts.

Regarding comparison tests, we are currently in the process of conducting comprehensive evaluations to quantify the benefits of our approach. These tests are designed to measure the improvement in relevance, coverage, user satisfaction, and the added value of consensus through redundancy against traditional single-query vector searches.

I anticipate publishing our findings soon and are optimistic about the potential implications for both academic research and practical applications.

Thanks again for your thoughtful query. It’s much appreciated and happy to answer any others questions.

Best, Adrian Raudaschl

On Thu, Mar 14, 2024 at 12:28 DavidGOrtega @.***> wrote:

👋

Interesting work! However I do not fully understand the need of generating multiple queries doing a vector search. Once the query is transformed into a vector the search is latent space centric so it will retrieve documents close to our vector. The rewritten queries will be semantically similar and I would not expect much benefit from this. The downside is the cost effectiveness having to multiply the number of queries and vector calculation for each query.

However, a completely different scenario would be generating the different queries in solely text based search DBs where having the same query reformulated with even different words that could retrieve semantically similar documents improving the results.

Have you made some comparison tests?

WDYT?

— Reply to this email directly, view it on GitHub https://github.com/Raudaschl/rag-fusion/issues/7, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABOSB5CWBFZCYAELMQZMIETYYGCV5AVCNFSM6AAAAABEV5EL7WVHI2DSMVQWIX3LMV43ASLTON2WKOZSGE4DMMJRGA4TCNQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[DavidGOrtega]

Hi Adrian,

thanks for your extremely well detailed response. Continuing with the discussion and possible implications. Once the vectors are generated should not suffice with one query aggregating the vectors averaging them or even better with attention? I mean, instead of doing 5 queries to each source we do 1 query with that aggregated vector to each source.

Also, if we are willing to be close to user intent should not the original query score more than the alternative ones?

[LucasLee-ff]

I think that when using not only vector search but also Elasticsearch or other methods, generating multiple queries would surely be a good way to increase recall.

By the way, your idea sounds interesting. Have you conducted any experiments on this?