search

microsoft / CodeXGLUE

CodeXGLUE
MIT License
1.56k stars 366 forks source link

How can I fine-tune CodeBERT on cloze test #127

Closed JaeWorld closed 2 years ago

JaeWorld commented 2 years ago

Hi. I'm trying to fine tune the CodeBERT model on cloze test, but I can't find any code for fine-tuning just like other tasks.

And as far as I know, there are no ground truth dataset provided for cloze test, which makes it impossible to do supervised learning.

Any way I can fine tune on cloze test in this situation?

celbree commented 2 years ago

As cloze test is a probing task similar with the pre-training task -- masked language modeling, it doesn't require fine-tuning. You can directly try CodeBERT on it.