search

awslabs / pptod

Multi-Task Pre-Training for Plug-and-Play Task-Oriented Dialogue System (ACL 2022)
https://arxiv.org/abs/2109.14739
Apache License 2.0
158 stars 27 forks source link

About PPTOD-large in End2End Modeling #9

Closed ShaneTian closed 2 years ago

ShaneTian commented 2 years ago

Thanks for your outstanding work!

In the End2End evaluation, the PPTOD-large is worse than PPTOD-small and PPTOD-base on both full-training (Table 2) and few-shot (Table 3) versions.

To a certain extent, this result is contrary to common sense. Is there any possible reason or experimental discovery?

yxuansu commented 2 years ago

Thanks for your outstanding work!

In the End2End evaluation, the PPTOD-large is worse than PPTOD-small and PPTOD-base on both full-training (Table 2) and few-shot (Table 3) versions.

To a certain extent, this result is contrary to common sense. Is there any possible reason or experimental discovery?

Hi,

Many thanks for your interest in our work!

For the PPTOD-large's performance in E2E modelling, we provide some analysis in our ACL camera-ready version which is available here (https://arxiv.org/pdf/2109.14739.pdf). Please find our analysis in section 4.1.3.

Please let me know if you have further questions :-)