InternEvo is an open-sourced lightweight training framework aims to support model pre-training without the need for extensive dependencies.
285
stars
47
forks
source link
fix(launch): remove use_paked_data=use_flash_atten assert #297
Closed
yingtongxiong closed 3 weeks ago
Remove the assertion in launch.py, when use_packed_data is False, the flash attention is also can enabled.