search

NVIDIA / Megatron-LM

Ongoing research training transformer models at scale
https://docs.nvidia.com/megatron-core/developer-guide/latest/user-guide/index.html#quick-start
Other
9.86k stars 2.23k forks source link

Does Megatron has plan to support Gemma？ #707

Open anlongfei opened 6 months ago

anlongfei commented 6 months ago

Your question Does Megatron has plan to support Gemma？

ethanhe42 commented 5 months ago

nemo (using megatron-core) supports it https://github.com/NVIDIA/NeMo/blob/f005f1323eaf9a23ad6dc4bc326dc95bf0002e8d/examples/nlp/language_modeling/conf/megatron_gemma_config.yaml#L4

fwyc0573 commented 5 months ago

also looking for example of gemma in megatron-lm

github-actions[bot] commented 3 months ago

Marking as stale. No activity in 60 days.