Fixing `test_evaluation` logic bugs

Future-House / paper-qa

High accuracy RAG for answering questions from scientific documents with citations

Apache License 2.0

6.44k stars 617 forks source link

Closed jamesbraza closed 1 week ago

jamesbraza commented 1 week ago

There were two problems with this test:

https://github.com/Future-House/paper-qa/pull/475 incorrectly moved from isinstance to > 0 checks, forgetting to delete the isinstance part
This mix up masked a bad change in https://github.com/Future-House/paper-qa/pull/494 where I mispropagated the index name

After this PR, we will correctly evaluate the stub-litqa dataset