Open lzc-nazarite opened 5 months ago
Apologies for the delayed reply. Which specific model are you training? What is the dtype? I haven't come across this particular issue before. You can contact me via email at anglv@ruc.edu.cn, which might be a more convenient way for discussion.
hello, i reproduce your mlm_run.py code and encounter some problems. -- 1. the loss appears 0 in the very early stage of training (actually in the epoch1) -- 2. during the evaluation, the metrics of trained mlm model are 0.
can you help me explain this phenomenon?