Closed seanmacavaney closed 2 years ago
@heinrichreimer - here's the related issue
@heinrichreimer -- interested in adding the touche queries/qrels in too?
Sure. I'm going to talk to Sascha directly regarding the different datasets.
Awesome, thanks! I'm keen to get these added, so let me know if I can be of assistance.
Dataset Information:
For argument retrieval (Task 1)
Links to Resources:
Dataset ID(s):
clef-touche/v1
(docs)clef-touche/v1/task1-2020
(docs, queries, qrels)clef-touche/2020-04-01
(docs)clef-touche/2020-04-01/task1-2020
(docs, queries, qrels) -- note: this is a distinct set of labels fromclef-touche/v1/task1-2020
-- refer to email "touché 2020 task 1 qrels" for detailsclef-touche/2020-04-01/task1-2021
(docs, queries, qrels) -- includes both quality and relevance judgmentsSupported Entities
Additional comments/concerns/ideas/etc.
Refer to email "touché 2020 task 1 qrels" for more details about the various versions of the qrels at play here. Specifically, there are separate qrels used for
beir/webis-touche
-- in the old version, a couple of qrels were missing, and the new version maps the revised2020-04-01/task1-2020
qrels onto thev1
corpus. IRDS seems like a good place to document all this.