search

TharinduDR / FT5

Apache License 2.0
1 stars 0 forks source link

Is it pre-training T5 from scratch? #1

Open tarudesu opened 9 months ago

tarudesu commented 9 months ago

Could I ask that is your work aiming to pre-train FT5 (T5-based model) from scratch, or just train on offensive tasks from T5 checkpoints?

@TharinduDR

tarudesu commented 9 months ago

Also, I'm curious about whether the pre-trained FT5 you proposed was trained from scratch or continually trained from the pre-trained T5 checkpoint. And which one are the results in the publication? (the one from scratch or the continuing trained).