search

amazon-science / RefChecker

RefChecker provides automatic checking pipeline and benchmark dataset for detecting fine-grained hallucinations generated by Large Language Models.
Apache License 2.0
281 stars 24 forks source link

Typo in claude2_extractor.py prompt #3

Closed sambayless closed 9 months ago

sambayless commented 9 months ago

Hi! RefChecker looks really neat!

I noticed there is a typo in the claude2_extractor prompt:

Now geneate the KG for the provided input text:

(At line 75, from here https://github.com/amazon-science/RefChecker/blob/main/refchecker/extractor/claude2_extractor.py)

"geneate" should be "generate"

sambayless commented 9 months ago

Also, from line 59 in the same file, "occured" should be "occurred":

("Announcement of Optimus", "occured at", "Artificial Intelligence (AI) Day event")

Both typos are part of the LLM prompt itself, so they potentially impact the LLM behaviour.

HuXiangkun commented 9 months ago

Thank you for pointing this out, we have corrected them accordingly.