laekov / fastmoe

A fast MoE impl for PyTorch
https://fastmoe.ai
Apache License 2.0
1.57k stars 189 forks source link

fastmoe是否支持微调,page-attention,flasahattention和kvcache,混合精度等 #192

Open PowerDispatch opened 9 months ago

PowerDispatch commented 9 months ago

Is your feature request related to a problem? Please describe. A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like A clear and concise description of what you want to happen.

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Additional context Add any other context or screenshots about the feature request here.

laekov commented 9 months ago

是, 是, 是和是, 是...

PowerDispatch commented 9 months ago

没有找到微调的case, 能帮忙引导一下吗

laekov commented 9 months ago

我们没有在仓库中包含微调的例子. 请您自行实现.

PowerDispatch commented 9 months ago

好的,谢谢