search

microsoft / IRNet

An algorithm for cross-domain NL2SQL
MIT License
268 stars 80 forks source link

Why is the lf_accuracy so low? #8

Closed xiaer1 closed 4 years ago

xiaer1 commented 4 years ago

This is my train log(the front 17 epoch): Epoch: 1, Loss: 12.740511, Acc: 0.032101, time: 123.496223 Epoch: 2, Loss: 6.106339,Acc: 0.050584, time: 134.791753 Epoch: 3, Loss: 4.475008, Acc: 0.050584, time: 153.214200 Epoch: 4, Loss: 3.618750, Acc: 0.055447, time: 168.053908 Epoch: 5, Loss: 3.046593, Acc: 0.065175, time: 182.702742 Epoch: 6, Loss: 2.570832, Acc: 0.066148, time: 198.103145 Epoch: 7, Loss: 2.241890, Acc: 0.061284, time: 210.114276 Epoch: 8, Loss: 1.898860, Acc: 0.060311, time: 239.237827 Epoch: 9, Loss: 1.692311, Acc: 0.062257, time: 241.019557 Epoch: 10, Loss: 1.526351, Acc: 0.069066, time: 278.005775 Epoch: 11, Loss: 1.324829, Acc: 0.072957, time: 286.218282 Epoch: 12, Loss: 1.185413, Acc: 0.071984, time: 276.896780 Epoch: 13, Loss: 1.124110, Acc: 0.067121, time: 284.550130 Epoch: 14, Loss: 1.017929, Acc: 0.059339, time: 303.098995 Epoch: 15, Loss: 0.913161, Acc: 0.066148, time: 321.235667 Epoch: 16, Loss: 0.824031, Acc: 0.060311, time: 341.108282 Epoch: 17, Loss: 0.755057, Acc: 0.061284, time: 356.403827 ... I hardly changed anything,what's wrong?

JasperGuo commented 4 years ago

Thanks for your report. We have made a mistake in measuring the accuracy and fixed the bug in this commit 3f8224514c21f50fb50c89d2730d8eb7ab0be15d . Please check it out and re-train the model.

shubhamk16 commented 4 years ago

hello @JasperGuo , I am getting acc, sketch acc =0, why so?

my training log looks like this sketch_correct and total and rule_label_correct::::>> 0 1 0 Epoch: 1, Loss: 37.402540, Sketch Acc: 0.000000, Acc: 0.000000, time: 2.477671

accuracy:::>> 0.0 0.0 sketch_correct and total and rule_label_correct::::>> 0 1 0 Epoch: 2, Loss: 32.059549, Sketch Acc: 0.000000, Acc: 0.000000, time: 1.529376

accuracy:::>> 0.0 0.0 sketch_correct and total and rule_label_correct::::>> 0 1 0 Epoch: 3, Loss: 26.147233, Sketch Acc: 0.000000, Acc: 0.000000, time: 1.581231

accuracy:::>> 0.0 0.0 sketch_correct and total and rule_label_correct::::>> 0 1 0 Epoch: 4, Loss: 22.054344, Sketch Acc: 0.000000, Acc: 0.000000, time: 1.937295