FreeLB-RoBERTa within HuggingFace's transformers?

zhuchen03 / FreeLB

Adversarial Training for Natural Language Understanding

250 stars 41 forks source link

FreeLB-RoBERTa within HuggingFace's transformers? #2

Closed zhengwsh closed 4 years ago

zhengwsh commented 4 years ago

Can you provide the FreeLB-RoBERTa in HuggingFace's transformers

zhuchen03 commented 4 years ago

Due to the difference in the implementations of fairseq and HuggingFace's, we have not spent time tuning the hyperparameters for the HuggingFace's version... We will handle the dropout mask for RoBERTa and release it soon!