MiuLab / SlotGated-SLU

Slot-Gated Modeling for Joint Slot Filling and Intent Prediction
303 stars 108 forks source link

Better performance could be achieved with larger patience #2

Open zhhongzhi opened 6 years ago

zhhongzhi commented 6 years ago

Thanks for sharing the codes. I run the model on ATIS with larger max-epoch number and patience value, and the result is much better than the result 82.5% which is reported in the paper. So I write to know why don't you report the better result.

2018-06-21 10:37:47,961 : INFO : Step: 8680
2018-06-21 10:37:47,961 : INFO : Epochs: 31 2018-06-21 10:37:47,961 : INFO : Loss: 0.017756752894118627 2018-06-21 10:37:48,120 : INFO : Valid: 2018-06-21 10:37:48,325 : INFO : slot f1: 97.34306569343066 2018-06-21 10:37:48,325 : INFO : intent accuracy: 97.6 2018-06-21 10:37:48,325 : INFO : semantic error(intent, slots are all correct): 89.8 2018-06-21 10:37:48,326 : INFO : Test: 2018-06-21 10:37:48,634 : INFO : slot f1: 95.4809213996835 2018-06-21 10:37:48,634 : INFO : intent accuracy: 94.8488241881299 2018-06-21 10:37:48,634 : INFO : semantic error(intent, slots are all correct): 85.1063829787234

YuanTingHsieh commented 6 years ago

I am able to reproduce the result similar to above

2018-06-21 14:28:53,183 : INFO : Step: 11200 2018-06-21 14:28:53,183 : INFO : Epochs: 40 2018-06-21 14:28:53,183 : INFO : Loss: 0.016021183816025898 2018-06-21 14:28:53,420 : INFO : Valid: 2018-06-21 14:28:54,005 : INFO : slot f1: 97.7725674091442 2018-06-21 14:28:54,005 : INFO : intent accuracy: 96.6 2018-06-21 14:28:54,005 : INFO : semantic error(intent, slots are all correct): 89.60000000000001 2018-06-21 14:28:54,006 : INFO : Test: 2018-06-21 14:28:54,868 : INFO : slot f1: 95.52924544972609 2018-06-21 14:28:54,868 : INFO : intent accuracy: 95.52071668533034 2018-06-21 14:28:54,868 : INFO : semantic error(intent, slots are all correct): 85.1063829787234