search

astronomer / ask-astro

An end-to-end LLM reference implementation providing a Q&A interface for Airflow and Astronomer
https://ask.astronomer.io/
Apache License 2.0
192 stars 47 forks source link

feat(api): MultiQueryRetriever with WeaviateHybridSearchRetriever #153

Closed Lee-W closed 10 months ago

Lee-W commented 10 months ago

Contradict to our earlier discussion; I suspect what we want to replace WeaviateHybridSearchRetriever with is MultiQueryRetriever instead of ConversationalRetrievalChain. However, it seems to perform even worse than the MultiQueryRetriever when it comes to what is astro python sdk?. astronomer-providers, airflow things seem to work. For testing, we can use the same set of env var and the exact same way to bring up the test env locally.

cloudflare-workers-and-pages[bot] commented 10 months ago

Deploying with  Cloudflare Pages  Cloudflare Pages

Latest commit: a0e9816
Status: ✅  Deploy successful!
Preview URL: https://4c2ded61.ask-astro.pages.dev
Branch Preview URL: https://change-number-of-doc.ask-astro.pages.dev

View logs

sunank200 commented 10 months ago

@Lee-W Now that we have MultiQueryRetriever with WeaviateHydridSearchRetriever. We should try the following two at the earliest when you start the day as Rahul needs to run the test for these.

We should try the following:

Let me know if you need any you need to connect on this, please

Lee-W commented 10 months ago

Increase the number of documents we retrieved per rewording to 5

tested with 6. a bit better but might have room to improve

when joining the docs together, we should rerank using something like cohere reranker

not yet have time to work on it today. will continue work on it early tomorrow

Lee-W commented 10 months ago

close this one, as changes included in https://github.com/astronomer/ask-astro/pull/170