search

epfLLM / Megatron-LLM

distributed trainer for LLMs
Other
504 stars 73 forks source link

Does it support sequence parallel? #101

Closed NamrataRShivagunde closed 4 months ago

martinjaggi commented 4 months ago

same as megatronLM base code. we did not touch the corresponding code