fix: split text by any newline and spaces

aws / fmeval

Foundation Model Evaluations Library

http://aws.github.io/fmeval

Apache License 2.0

155 stars 40 forks source link

fix: split text by any newline and spaces #178

Closed franluca closed 5 months ago

franluca commented 5 months ago

Description of changes: when computing f1 measures for QA, split the text by any number of newlines or spaces (previously was considering only one new line).

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

franluca commented 5 months ago

Made changes to documentation, added a test case and corrected lint.

I think it's fine to have one separate function for split even if it calls only str.split . This adds clarity to the code.