search

aliyun / aicb

Other
125 stars 19 forks source link

Update megatron_gpt.sh #6

Open 1195343015 opened 2 months ago

1195343015 commented 2 months ago

fix bugs

CLAassistant commented 2 months ago

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

zhouheyang-alibaba commented 2 months ago

Here, the issue with the default value of seq_length has been fixed, but why should we default to disabling aiob?

1195343015 commented 2 months ago

Here, the issue with the default value of seq_length has been fixed, but why should we default to disabling aiob?

https://github.com/aliyun/aicb/blob/0877f86479041a1d04ff2803c59413cdc43b93df/scripts/megatron_gpt.sh#L173

This line arouses bug. Maybe need to change this line instead of disabling aiob

1195343015 commented 2 months ago

bugs could be reproduced by sh scripts/megatron_gpt.sh -m 7.