search

karpathy / build-nanogpt

Video+code lecture on building nanoGPT from scratch
3.14k stars 401 forks source link

Integrating GPT-2 with deepspeed Zero-1, Zero-2 and Zero-3 #48

Open Devadeut opened 1 month ago

dustinwloring1988 commented 3 weeks ago

Cant you run this with a custom config for accelerate?