search

openlm-research / open_llama

OpenLLaMA, a permissively licensed open source reproduction of Meta AI’s LLaMA 7B trained on the RedPajama dataset
Apache License 2.0
7.29k stars 372 forks source link

I didn't find dropout in the paper. May I ask if llama has used dropout? #22

Closed joytianya closed 1 year ago

young-geng commented 1 year ago

I don't think llama used dropout during pretraining

joytianya commented 1 year ago

thank you ~