QData / TextAttack

TextAttack πŸ™ is a Python framework for adversarial attacks, data augmentation, and model training in NLP https://textattack.readthedocs.io/en/master/
https://textattack.readthedocs.io/en/master/
MIT License
2.96k stars 396 forks source link

Custom fine-tuned model gets weird results #803

Open penguinwang96825 opened 2 months ago

penguinwang96825 commented 2 months ago

I'm experiencing unusual results with the custom fine-tuned model textattack/roberta-base-SST-2. The clean accuracy is significantly lower than expected, at 50.69%. I've included the attack results below, which show a low accuracy under attack and a high number of skipped attacks. I'd appreciate it if you could look into this issue.

ζˆͺεœ– 2024-09-06 δΈ‹εˆ4 02 10
penguinwang96825 commented 2 months ago

I'm also encountering an issue with the bert-base-uncased model fine-tuned on several datasets. It suddenly behaves abnormally. I've tried reinstalling TextAttack, but the problem persists.