search

hpcaitech / Open-Sora

Open-Sora: Democratizing Efficient Video Production for All
https://hpcaitech.github.io/Open-Sora/
Apache License 2.0
21.57k stars 2.07k forks source link

[HELP] In my train OpenSora1.0 process， I got zhe error ['LowLevelZeroOptimizer' object has no attribute '_param_store'], how to fix it？ #642

Closed pqhgit closed 2 weeks ago

pqhgit commented 1 month ago

Traceback (most recent call last): File "/xxx/OpenSora1.0/scripts/train.py", line 328, in main() File "/xxx/OpenSora1.0/scripts/train.py", line 270, in main update_ema(ema, model.module, optimizer=optimizer) File "/usr/local/python-3.10.13/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/xxx/OpenSora1.0/opensora/utils/train_utils.py", line 31, in update_ema master_param = optimizer._param_store.working_to_master_param[param_id] AttributeError: 'LowLevelZeroOptimizer' object has no attribute '_param_store'

I can't find any class with 'LowLevelZeroOptimizer' or '_param_store' in this project, how to fix it？

narrowsnap commented 1 month ago

In my experiments, I change it to master_param = optimizer.working_to_master_param[param_id].

pqhgit commented 1 month ago

@narrowsnap this worked, thx

pqhgit commented 1 month ago

modify /xxx/OpenSora1.0/opensora/utils/train_utils.py", line 31 -> master_param = optimizer.working_to_master_param[param_id] solved.

FlyHighest commented 1 month ago

pip install colossalai==0.4.0

github-actions[bot] commented 3 weeks ago

This issue is stale because it has been open for 7 days with no activity.

github-actions[bot] commented 2 weeks ago

This issue was closed because it has been inactive for 7 days since being marked as stale.

Texaser commented 2 weeks ago

After changing it to master_param = optimizer.working_to_master_param[param_id], I still have the AttributeError: : 'LowLevelZeroOptimizer' object has no attribute 'working_to_master_param'. Can anyone help?

narrowsnap commented 2 weeks ago

After changing it to master_param = optimizer.working_to_master_param[param_id], I still have the AttributeError: : 'LowLevelZeroOptimizer' object has no attribute 'working_to_master_param'. Can anyone help?

What's your version of colossalai?