Test and evaluate the quality of response after Astronomer providers data ingestion

astronomer / ask-astro

An end-to-end LLM reference implementation providing a Q&A interface for Airflow and Astronomer

https://ask.astronomer.io/

Apache License 2.0

192 stars 47 forks source link

Test and evaluate the quality of response after Astronomer providers data ingestion #185

Closed sunank200 closed 9 months ago

sunank200 commented 10 months ago

After the data is ingested along with astronomer provider docs we need to run the same test plan again and evaluate the results.

Depends on: #166

vatsrahul1001 commented 9 months ago

I have tested Astronomer Providers' questions, and I can confirm that some of the responses have improved. However, basic questions like 'Astronomer provider version' and 'What is Astronomer Providers' are still incorrect. There is certainly room for improvement in this area.

Results