search

microsoft / CodeBERT

CodeBERT
MIT License
2.23k stars 454 forks source link

CodeBERT Pre-trained models trained from scratch. #171

Closed donhk1011 closed 2 years ago

donhk1011 commented 2 years ago

Hi! Could you please let me know where I can find the pre-trained models from scratch (random initialization)? Specifically, I am looking for the checkpoints of PT W/ CODE ONLY (INIT=S) and CODEBERT (MLM, INIT=S) in Table 2?

Thanks!

guoday commented 2 years ago

Sorry, we don't save this model.

donhk1011 commented 2 years ago

Thanks for the reply! Is it possible to get a training code for this?

guoday commented 2 years ago

https://github.com/huggingface/transformers/tree/main/examples/pytorch/language-modeling#robertabertdistilbert-and-masked-language-modeling

donhk1011 commented 2 years ago

Thanks a lot for the pointer!