How to perform error analysis

soyeonm / FILM

Official repository of ICLR 2022 paper FILM: Following Instructions in Language with Modular Methods

116 stars 28 forks source link

How to perform error analysis #8

Open joeyy5588 opened 2 years ago

joeyy5588 commented 2 years ago

Hi, thanks for sharing this amazing work!

I've read your paper and in Table 3 (Section 5.2) you show the percentage of error mode of FILM. It would be great to know how to analyze the error like your paper did. Because I couldn't find such information in the pickle file generated under results/fails.

Thanks!

jiadingfang commented 1 year ago

second this, how do I automatically tell which error type the failed run belongs to? Can you provide some hint? @soyeonm