issues
search
volcengine
/
veScale
A PyTorch Native LLM Training Framework
http://vescale.xyz
Apache License 2.0
553
stars
26
forks
source link
[DTensor & DModule & DOptim] feature updates
#20
Closed
jc-bytedance
closed
4 months ago
jc-bytedance
commented
4 months ago
In this PR, we update some features in our DTensor & DModule & DOptim implementations, Yo~
DTensor Updates:
Support more dtensor ops.
Sharding Strategy Updates.
DModule Updates:
Decouple uneven support and run check.
Reduce some CPU overhead.
DOptim Updates:
More fridenly API.
Unit test updates.
Reorder some communication for better results.
Other Updates/fixes:
Some minor update on our nano GPT model and test results.
In this PR, we update some features in our DTensor & DModule & DOptim implementations, Yo~
DTensor Updates:
DModule Updates:
DOptim Updates:
Other Updates/fixes: