search

microsoft / AI-System

System for AI Education Resource.
https://microsoft.github.io/AI-System/
Creative Commons Attribution 4.0 International
3.63k stars 450 forks source link

Origin/lynex #94

Closed lynex closed 2 years ago

YanjieGao commented 2 years ago

Please add the related 2 labs in course to this chapter (add description text and link to lab folder) as experiment, template could follow chapter 7 and 8. This will boost the in action experience and book's system style.

https://github.com/microsoft/AI-System/tree/main/Labs/BasicLabs/Lab4

https://github.com/microsoft/AI-System/tree/main/Labs/AdvancedLabs/Lab7

Refer: https://github.com/microsoft/AI-System/blob/main/Textbook/%E7%AC%AC8%E7%AB%A0-%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E6%8E%A8%E7%90%86%E7%B3%BB%E7%BB%9F/8.4-%E9%83%A8%E7%BD%B2.md#846-%E6%8E%A8%E7%90%86%E7%B3%BB%E7%BB%9F%E9%83%A8%E7%BD%B2%E4%B8%8E%E5%8E%8B%E6%B5%8B%E5%AE%9E%E9%AA%8C%E7%BB%83%E4%B9%A0

YanjieGao commented 2 years ago

image 代码实例换为下面格式

code

，方便各章统一格式

YanjieGao commented 2 years ago

汉化这个表格 image

YanjieGao commented 2 years ago

chapter 6.5 还需要统一格式

YanjieGao commented 2 years ago

Google Pathway and MOE distributed training have new pattern? like Dynamic/ Sparse activation? 如果有新pattern是否需要加入到pipeline并行后做趋势对比和展望？

YanjieGao commented 2 years ago

好像缺少一块内容，task placement? （可以融合进算法对比或者框架或者新开section）例如在模型和pipeple并行里，这块有个优化问题，记得有google paper提过优化，以及有paper auto tuning这个过程。

YanjieGao commented 2 years ago

Hello 山哥，我们先merge一版再开新PR否？

YanjieGao commented 2 years ago

Please merge this to avoid long pending, please solve the conflict and solve the format issues.

YanjieGao commented 2 years ago

Please refine in next PR