Which dataset should we use for evaluate?

locuslab / tofu

Landing Page for TOFU

MIT License

98 stars 24 forks source link

Which dataset should we use for evaluate? #43

Open Yuda-Jin opened 2 months ago

Yuda-Jin commented 2 months ago

which dataset config was used in leaderboard? Should I use forget10_perturbed or just forget10 or retain90? If I use forget10 dataset, how to set perturbed_answer_key and eval_task?

Yuda-Jin commented 2 months ago

For more specific, which config was used in baselines in leaderboard?