Want to use zero3 to train KTO and met error

huggingface / trl

Train transformer language models with reinforcement learning.

http://hf.co/docs/trl

Apache License 2.0

8.61k stars 1.06k forks source link

Want to use zero3 to train KTO and met error #1770

Open Faded1022 opened 6 days ago

Faded1022 commented 6 days ago

lib/python3.10/site-packages/deepspeed/runtime/zero/stage3.py", line 435, in defragment assert len(set(t.dtype for t in tensors)) == 1 assert len(set(t.dtype for t in tensors)) == 1