The subset of questions parameter is not working previously for the DAG. It was not able to parse a list of question numbers taken in as input and causes errors (outlined in the issue linked below)
weaviate_search_multiquery_retriever is deleted: it is no longer relevant or useful. It was not a good practice to hardcode a way of doing multiquery retrieval in the evaluation run DAG since the retrieval chain in the api server can change anytime. This would not produce related result in such a scenario and do not add additional data insights. (E.g. right now we do multiquery differently, uses hybrid search and reranker, none of these are taken into account)
weaviate_search_references is deleted. Similar reason as above. The actual RAG pipeline now uses hybrid search + reranker. There's no reason to query for results from weaviate using an embedding model. It is not a comparable result.
Description
weaviate_search_multiquery_retriever
is deleted: it is no longer relevant or useful. It was not a good practice to hardcode a way of doing multiquery retrieval in the evaluation run DAG since the retrieval chain in the api server can change anytime. This would not produce related result in such a scenario and do not add additional data insights. (E.g. right now we do multiquery differently, uses hybrid search and reranker, none of these are taken into account)weaviate_search_references
is deleted. Similar reason as above. The actual RAG pipeline now uses hybrid search + reranker. There's no reason to query for results from weaviate using an embedding model. It is not a comparable result.Tests
closes https://github.com/astronomer/ask-astro/issues/298