TypeError in `orpo_trainer.train()`: 'str' object is not callable

[kdunee]

When running the ORPO Unsloth Example.ipynb notebook, I encountered an error during the execution of orpo_trainer.train(). The error occurs consistently across different GPU types and persists even with slightly older versions of unsloth and its dependencies.

Steps to Reproduce

1. Run the ORPO Unsloth Example.ipynb notebook
2. Execute all cells up to and including orpo_trainer.train()

Error Message

----> 1 orpo_trainer.train()

7 frames
/usr/local/lib/python3.10/dist-packages/transformers/trainer.py in train(self, resume_from_checkpoint, trial, ignore_keys_for_eval, **kwargs)
   1936                 hf_hub_utils.enable_progress_bars()
   1937         else:
-> 1938             return inner_training_loop(
   1939                 args=args,
   1940                 resume_from_checkpoint=resume_from_checkpoint,

/usr/local/lib/python3.10/dist-packages/unsloth/models/llama.py in _fast_inner_training_loop(self, batch_size, args, resume_from_checkpoint, trial, ignore_keys_for_eval)

/usr/local/lib/python3.10/dist-packages/accelerate/data_loader.py in __iter__(self)
    461                 # But we still move it to the device so it is done before `StopIteration` is reached
    462                 if self.device is not None:
--> 463                     current_batch = send_to_device(current_batch, self.device, non_blocking=self._non_blocking)
    464                 next_batch = next(dataloader_iter)
    465                 if batch_index >= self.skip_batches:

/usr/local/lib/python3.10/dist-packages/accelerate/utils/operations.py in send_to_device(tensor, device, non_blocking, skip_keys)
    181             skip_keys = []
    182         return type(tensor)(
--> 183             {
    184                 k: t if k in skip_keys else send_to_device(t, device, non_blocking=non_blocking, skip_keys=skip_keys)
    185                 for k, t in tensor.items()

/usr/local/lib/python3.10/dist-packages/accelerate/utils/operations.py in <dictcomp>(.0)
    182         return type(tensor)(
    183             {
--> 184                 k: t if k in skip_keys else send_to_device(t, device, non_blocking=non_blocking, skip_keys=skip_keys)
    185                 for k, t in tensor.items()
    186             }

/usr/local/lib/python3.10/dist-packages/unsloth/models/_utils.py in _fixed_send_to_device(tensor, device, non_blocking, skip_keys)

/usr/local/lib/python3.10/dist-packages/accelerate/utils/operations.py in honor_type(obj, generator)
     79         return type(obj)(*list(generator))
     80     else:
---> 81         return type(obj)(generator)
     82 
     83 

/usr/local/lib/python3.10/dist-packages/unsloth/models/_utils.py in <genexpr>(.0)

TypeError: 'str' object is not callable

Environment

• Google Colab
• GPU: Tested on both T4 and A100

==((====))==  Unsloth 2024.8: Fast Llama patching. Transformers = 4.44.2.
   \\   /|    GPU: Tesla T4. Max memory: 14.748 GB. Platform = Linux.
O^O/ \_/ \    Pytorch: 2.4.0+cu121. CUDA = 7.5. CUDA Toolkit = 12.1.
\        /    Bfloat16 = FALSE. FA [Xformers = 0.0.27.post2. FA2 = False]
 "-____-"     Free Apache license: http://github.com/unslothai/unsloth

Additional Information

• The error occurs consistently, even with a copy of the notebook using my own data.
• I tried using a slightly older version of unsloth and some older dependencies, but the issue persisted.

Possible Cause

The error seems to be related to the send_to_device function in the accelerate library, specifically when trying to move data to the GPU. It appears that somewhere in this process, the code is attempting to call a string object as if it were a function.

Any assistance in resolving this issue would be greatly appreciated. Let me know if you need any additional information or if you'd like me to run any specific tests.

[kdunee]

I tracked the issue down to unsloth/models/_utils.py patching accelerate.utils.operations.send_to_device.

For example, this code works:

import accelerate

accelerate.utils.operations.send_to_device({
    "commit": ["Hello",]
}, "gpu")

This code doesn't (same as above, TypeError: 'str' object is not callable):

import accelerate
from unsloth.models import _utils

accelerate.utils.operations.send_to_device({
    "commit": ["Hello",]
}, "gpu")

I ended up temporarily running this code at the beginning of my notebook to "unpatch" send_to_device:

import accelerate
orig = accelerate.utils.operations.send_to_device
from unsloth.models import _utils
accelerate.utils.operations.send_to_device = orig

[danielhanchen]

Will check and fix! Sorry on the issue!

[mlabonne]

I also get this error with the DPO Zephyr Unsloth Example (https://colab.research.google.com/drive/15vttTpzzVXv_tJwEk-hIcQ0S9FcEWvwP?usp=sharing). Interestingly, it doesn't raise this issue with the default "unsloth/zephyr-sft-bnb-4bit" but when I used my own model (mlabonne/TwinLlama-3.1-8B).

Applying @kdunee's "unpatch" at the beginning of the notebook fixed it for me (thanks!).

[danielhanchen]

Oh I commented out my overriding of accelerate - weirdly Xformers and Transformers work now - I added the patch because inference weirdly broke, so I had to add a try except inside of accelerate - hopefully DPO and ORPO work now! So sorry on the issue!

[danielhanchen]

For local machines, please update Unsloth via:

pip uninstall unsloth -y
pip install --upgrade --no-cache-dir "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"

[pgurazada]

@danielhanchen Thank you for this, the fix now works. Tested it extensively across my dpo code. No need of the "unpatch" from @kdunee anymore.

[mlabonne]

Thanks @danielhanchen!