【BUG】occur error：AttributerError：'DeepSpeedHybridEngine' object has no attribute 'mp_group' whiling run llama7b for step3/rlhf/ppo

microsoft / DeepSpeedExamples

Example models using DeepSpeed

Apache License 2.0

6.11k stars 1.04k forks source link

【BUG】occur error：AttributerError：'DeepSpeedHybridEngine' object has no attribute 'mp_group' whiling run llama7b for step3/rlhf/ppo #375

Open Pattaro opened 1 year ago

Pattaro commented 1 year ago

98DDB13F-60AE-4F7D-8979-9B287A2A4CC1

Chevolier commented 1 year ago

Any solutions? I encountered the same issue with bloomz model, so far I just removed --enable_hybrid_engine to bypass the issue, and the program runs. However, guess the efficiency would reduce.

DwarfWarriors commented 1 year ago

It seems enable_hybrid_engine only support for OPT models.

kkk935208447 commented 8 months ago

try adjusting the --inference_tp_size to a lower number, it may be you don't have enough GPUs across your nodes.

[bug]AttributeError: 'DeepSpeedHybridEngine' object has no attribute 'mp_group' #525