Closed KerenzaDoxolodeo closed 10 months ago
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.
Please note that issues that do not follow the contributing guidelines are likely to be ignored.
Hi @KerenzaDoxolodeo, thanks for raising an issue!
This is a question best placed in our forums. We try to reserve the github issues for feature requests and bug reports.
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.
Please note that issues that do not follow the contributing guidelines are likely to be ignored.
System Info
transformers
version: 4.33.0Who can help?
No response
Information
Tasks
examples
folder (such as GLUE/SQuAD, ...)Reproduction
I run xlm-roberta in three implementations:
1) Using TFAutoModelForSequenceClassification 2) Using TFAutoModel, with the classification layer as faithful as possible to huggingface's implementation.
Code : https://www.kaggle.com/code/realdeo/keras-code/settings?scriptVersionId=145530298
3) Using TrainerAPI
Code : https://www.kaggle.com/code/realdeo/fork-of-notebookcb67cb4ef2/notebook?scriptVersionId=145540775
Expected behavior
I expect the code to have roughly the same accuracy. What happens here is the Trainer API successfully trained after 1 epoch while the tensorflow implementation stuck at predicting the same label.