Closed RishabhMaheshwary closed 4 years ago
May I ask what is the "max_seq_length" you have set?
512
I checked my training code for the fake dataset and found that I used 256 as the "max_seq_length". I also double checked that the pre-trained model parameters can achieve 99.9% accuracy using this length setting.
For Infersent, I trained their model from scratch. But when I load the pretrained model, I get the following error:
RuntimeError: Error(s) in loading state_dict for NLINet: Missing key(s) in state_dict: "classifier.4.weight", "classifier.4.bias", "classifier.7.weight", "classifier.7.bias". Unexpected key(s) in state_dict: "classifier.0.weight", "classifier.0.bias", "classifier.2.weight", "classifier.2.bias". size mismatch for classifier.1.weight: copying a param with shape torch.Size([512, 512]) from checkpoint, the shape in current model is torch.Size([512, 16384]).
Edit: Solved by changing 'nonlinear_fc' to 0 in config.
Glad to hear that you have solved the error. I will close this issue, but if you have further questions, feel free to open it.
The orig_failures count is 408 on fake news BERT. Is the pretrained model you uploaded the correct one?