predict accuracy and eval accuracy

[YiLing28]

in data folder , i try to train and eval(ganbert), the eval result is: eval_accuracy = 0.516 eval_f1_macro = 0.1780866 eval_f1_micro = 0.516 eval_loss = 2.249187 eval_precision = 0.516 eval_recall = 0.516 global_step = 276 loss = 5.5095177 but when i try to predict the test.txt, the accuracy is low,maybe 0.1.( i set do_predict=true, and set LABEL_RATE="1") how to predict?

[louisowen6]

Same issue here. I've used the trained model checkpoint and try to predict on test.tsv. However, the result different from the eval_accuracy (~0.5 for eval but ~0.1 for test), although the eval_accuracy is also extracted from the evaluation on the same test.tsv.

Update: I found the issue. It's because of the random.shuffle(to_write_examples) located in line 291 ganbert.py. I suggest adding if-else, so it only does the shuffle on train/validation but not on the prediction task.

[kamei86i]

Hi,

thanks @louisowen6 for the discovery of the problem! We will make soon a revision of the code!