baichuan-inc / Baichuan-7B

A large-scale 7B pretraining language model developed by BaiChuan-Inc.
https://huggingface.co/baichuan-inc/baichuan-7B
Apache License 2.0
5.67k stars 506 forks source link

[Question] deepspeed 无法使用zero3 #71

Open expresschen opened 1 year ago

expresschen commented 1 year ago

Required prerequisites

Questions

image 当时使用zero启动时出现上图中的错误

Checklist

wqmoran commented 1 year ago

我也遇到这个问题了,想问问最后是怎么解决的呢?

expresschen commented 1 year ago

我是升级deepspeed的版本到0.9.5,错误就消失啦