astronomer / ask-astro

An end-to-end LLM reference implementation providing a Q&A interface for Airflow and Astronomer
https://ask.astronomer.io/
Apache License 2.0
192 stars 47 forks source link

Test and evaluate the result for ingested data from Airflow docs with tokenization as lowercase #144

Closed sunank200 closed 10 months ago

sunank200 commented 10 months ago

After the data is ingested along with airflow docs with tokenization as lowercase we need to run the same test plan again and evaluate the results.

Depends on: #145

vatsrahul1001 commented 10 months ago

This is not Priority for @28th release as discussed with @sunank200 , we can work on this post release

vatsrahul1001 commented 10 months ago

Tested completed on this, I do not see any degradation due to this. Overall responses arr sources are fine

Results: https://docs.google.com/spreadsheets/d/13cVqNikix82YjCPA4t0XaULg3XccBnvrQUmQa9VwgC0/edit#gid=2063029621