Closed ghaddarAbs closed 2 years ago
Hi,
I just wanted to ask if you pretrained your model on the next sentence prediction task ?
Thanks
We followed the exact pretraining objectives introduced in the BERT paper. We pretrained using both MLM and NSP.
Thanks, it answer my question ... I will close the issue
Hi,
I just wanted to ask if you pretrained your model on the next sentence prediction task ?
Thanks