[Example] add an example of running open llama model in 4D using veScale

volcengine / veScale

A PyTorch Native LLM Training Framework

http://vescale.xyz

Apache License 2.0

553 stars 26 forks source link

[Example] add an example of running open llama model in 4D using veScale #22

Closed jc-bytedance closed 4 months ago

jc-bytedance commented 4 months ago

This PR adds an 4D parallelism example of using veScale to run a open llama model that is directly imported from HuggingFace without any model code modifications.