NanoDL 1.2.0.dev1 - Githubissues

HMUNACHI / nanodl

A Jax-based library for designing and training transformer models from scratch.

MIT License

272 stars 11 forks source link

Closed HMUNACHI closed 6 months ago

HMUNACHI commented 6 months ago

Google's Gemma architecture.
Reward model wrapper and data-parallel distributed reward trainer.
True random number generators in Jax which do not need the verbose code (examples shown in next sections).