search

google / maxtext

A simple, performant and scalable Jax LLM!
Apache License 2.0
1.39k stars 247 forks source link

Prefill return first token #727

Closed jwyang-google closed 3 weeks ago

jwyang-google commented 4 weeks ago

Modify Jetstream to make prefill return first token. Need to check in together with https://github.com/google/JetStream/pull/105