How to use it? - Githubissues

PKU-DAIR / Hetu-Galvatron

Galvatron is an automatic distributed training system designed for Transformer models, including Large Language Models (LLMs).

34 stars 3 forks source link

How to use it? #1

Open robertLiuLinFeng opened 9 months ago

robertLiuLinFeng commented 9 months ago

"Could you please tell me how to use this library? I don't see the construct_parallel_models() API."

Fizzmy commented 1 week ago

You can relate to Galvatron Model Usage for more details. The construct_parallel_models() API is named construct_hybrid_parallel_model_api in each model. It can be found in galvatron/models/MODEL_TYPE/MODEL_TYPE_hybrid_parallel.py.