Closed ljvmiranda921 closed 1 year ago
Maybe...follow how the google paper constructs their chain of thought prompts? Current unknowns:
I remember an idea that annotator disagreements are also nice signals (of what?). Maybe there's something there.
How about, in the future, use langchain to feed an annotation guideline (ACE2004/The Guardian), then let the LLM show the exact passage as to why that was the case. Then you can kinda verify it