search

FullFact / health-misinfo-shared

Raphael health misinformation project, shared by Full Fact and Google
MIT License
0 stars 0 forks source link

Ask for a direct quote as well as a paraphrase in multi label prompt #95

Closed dearden closed 4 months ago

dearden commented 4 months ago

Overview

The main prompt we are using is HEALTH_INFER_MULTI_LABEL_PROMPT. This only asks for claims, and does not specifically ask for both a paraphrased claim and a direct quote.

Asking for a direct quote too would help us...

  1. Evaluate whether the paraphrases are accurate and match the quote
  2. Evaluate whether the direct quote is being hallucinated
  3. Find more specific timestamps for the claim within the text (because we could go from direct quote to specific line in the transcript)

Requirements

  1. A part of the prompt that asks for a direct quote and paraphrase
  2. Otherwise the output remains the same
  3. Paraphrase will have the same field name as the claim in the current system