AppraiseDev / Appraise

Appraise code used as part of WMT21 human evaluation campaign
BSD 3-Clause "New" or "Revised" License
22 stars 12 forks source link

MQM/ESA separate tutorials #147

Closed zouharvi closed 7 months ago

zouharvi commented 7 months ago

A few points by @kocmitom that point to the tutorial having to be distinct between MQM and ESA:

The result is that we'll have to have distinct batches for ESA and MQM because the tutorial is data.

zouharvi commented 7 months ago

For ESA:

image

image

zouharvi commented 7 months ago

For MQM:

image

image

zouharvi commented 7 months ago

The instructions say that the users need to set the score.

image

zouharvi commented 7 months ago

Resolved in 7c9cf88.

kocmitom commented 7 months ago

I would mention in each ESA step "and move slider to X". Since they need to do two steps

zouharvi commented 7 months ago

I don't understand. What do you mean by "in each ESA step"? Are you suggesting to edit the instructions? How exactly?

zouharvi commented 7 months ago

Got it from our chat.

image

image