code for evaluation - Githubissues

jzbjyb / FLARE

Forward-Looking Active REtrieval-augmented generation (FLARE)

MIT License

545 stars 50 forks source link

code for evaluation #10

Open zthang opened 10 months ago

zthang commented 10 months ago

Good work! Can you provide the evaluation code for the results presented in the paper (exactly math, f1, recall and precision)? Thanks!

jennybae1024 commented 10 months ago

same request here! cannot reproduce the result on the paper when evaluating by whether the lowercase version of the output is in the lower case of the gold answer.

LLSuzy commented 3 months ago

Line 131 "examples = self_consistency(examples, anchor_text=anchor_text)" in prep.py file. This line reports an error. The error message is that the self_consistency function is not defined. Can you please tell me if the code for this function is publicly available. I can't find this function in any of the files.