bclavie
commented
1 month ago Hey,
I'm not sure I understand your request correctly -- this repo isn't MosaicBERT but a separate effort to continue modernising the architecture & train on "modern" volumes of data.
We're definitely planning on releasing our checkpoints once training is over 😄
Hi, Thanks for the great work I consider fine-tuning moisacbert on tasksource https://github.com/sileod/tasksource/ The deberta-tasksource version is quite popular But do you plan to release updates on moisacbert ? e.g. longer training