Change the langchain code for Hybrid search

astronomer / ask-astro

An end-to-end LLM reference implementation providing a Q&A interface for Airflow and Astronomer

https://ask.astronomer.io/

Apache License 2.0

196 stars 47 forks source link

Closed sunank200 closed 12 months ago

sunank200 commented 1 year ago

As the response for Ask-Astro is off for certain responses as described in #133

We should try the following:

increase the number of documents we retrieve per rewording
when joining the docs together, we should rerank using something like cohere reranker
it seems like the documents are all in weaviate properly, so it feels like a problem with how we do document retrieval. We could also try hybrid searchhttps://python.langchain.com/docs/integrations/retrievers/weaviate-hybrid

phanikumv commented 12 months ago

PR has been created for this.

Lee-W commented 12 months ago

As we've been testing with this, and doesn't seem to work, should we close this one and related PRs?

sunank200 commented 12 months ago

Yes we should close this as it won't go in for Nov 28th release @Lee-W

phanikumv commented 12 months ago

what about the first two tasks?

increase the number of documents we retrieve per rewording
when joining the docs together, we should rerank using something like cohere reranker

Lee-W commented 12 months ago

sunank200 commented 12 months ago

increase the number of documents we retrieve per rewording

when joining the docs together, we should rerank using something like cohere reranker

Both are done in this #170