search

smartyfh / MultiWOZ2.4

MultiWOZ 2.4: A Multi-Domain Task-Oriented Dialogue Dataset
MIT License
60 stars 7 forks source link

which results of CHAN-DST should we use? #2

Open fpcsong opened 3 years ago

fpcsong commented 3 years ago

I notice that CHAN-DST and STAR are both in your git repo, and in paper of STAR you rerun the CHAN and report its joint goal as 53.38% at multi-woz2.1, and in its original paper it was 58.55% at multiwoz2.1, so which is offical?

smartyfh commented 3 years ago

I notice that CHAN-DST and STAR are both in your git repo, and in paper of STAR you rerun the CHAN and report its joint goal as 53.38% at multi-woz2.1, and in its original paper it was 58.55% at multiwoz2.1, so which is offical?

The code was forked from the authors' repo. But I rerun the experiments using the same preprocessing as STAR.

fpcsong commented 3 years ago

I also failed to re-produce "58.55%" and I notice that CHAN-DST has been withdrawn from ACL 2020, so be it. :joy: