AI-Hypercomputer / maxtext

A simple, performant and scalable Jax LLM!
Apache License 2.0
1.47k stars 275 forks source link

Add load balance loss #860

Closed RissyRan closed 3 weeks ago

RissyRan commented 3 weeks ago

Description

Test

local test on a small model size: