How to train from scratch? Can you provide the code

[sankexin]

System Info

train from scratch

Information

• [ ] The official example scripts
• [ ] My own modified scripts

Tasks

• [ ] An officially supported task in the examples folder
• [ ] My own task or dataset (give details below)

Reproduction

train from scratch

Expected behavior

train from scratch

Checklist

• [X] I have checked that my issue isn't already filed (see open issues)
• [X] I have included my system information
• [X] Any code provided is minimal, complete, and reproducible (more on MREs)
• [X] Any code provided is properly formatted in code blocks, (no screenshot, more on code blocks)
• [X] Any traceback provided is complete

[qgallouedec]

Does "from scratch" means the opposite of "finetuning" for you? Please precise your question

[kalocide]

why would you pre-train with RL?