search

LLM360 / amber-train

Pre-training code for Amber 7B LLM
Apache License 2.0
141 stars 15 forks source link

Where did you train the model? #3

Closed filopedraz closed 5 months ago

filopedraz commented 5 months ago

Did you use Deepspeed to train the model?

hunterhector commented 5 months ago

The backend is simply FSDP, https://github.com/LLM360/amber-train/blob/main/main.py#L129