search

SupritYoung / KBert-En

I modified some code of K-BERT so that it can be fit to English datasets Topics Resources
9 stars 1 forks source link

Adapting K-BERT for other BERT-based models #2

Open vishprivenkat opened 1 year ago

vishprivenkat commented 1 year ago

Hi, I was trying to adapt K-BERT for RoBERTa and tried using the pre-trained model for RoBERTa from Huggingface for that. But somehow, the model never seems to converge at all and gives very poor scores. Could you please guide me on how to adapt K-BERT for another BERT-based model?

Thank You!

SupritYoung commented 1 year ago

U need to modify the code related to PLM according to Huggingface doc, such as the parameters in model function etc.