I am on the latest MacOs (13.4 (22f66)) on a MacBook air m2 16gb. I installed the opt6.7B version whilst choosing to run on CPU (also tried apple silicon, got same results). The issue occurs when I am in the localhost and when I press "Generate" with any prompt, I also enabled --mlock option, no other differences from default configuration.
What can cause this issue, and are there any fixes for it?
Traceback of the issue:
/Users/sadesguy/Downloads/one-click-installers-main/oobabooga_macos/installer_files/env/lib/python3.10/site-packages/transformers/generation/utils.py:719: UserWarning: MPS: no support for int64 repeats mask, casting it to int32 (Triggered internally at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1682343686130/work/aten/src/ATen/native/mps/operations/Repeat.mm:236.)
input_ids = input_ids.repeat_interleave(expand_size, dim=0)
Traceback (most recent call last):
File "/Users/sadesguy/Downloads/one-click-installers-main/oobabooga_macos/text-generation-webui/modules/callbacks.py", line 73, in gentask
ret = self.mfunc(callback=_callback, **self.kwargs)
File "/Users/sadesguy/Downloads/one-click-installers-main/oobabooga_macos/text-generation-webui/modules/text_generation.py", line 277, in generate_with_callback
shared.model.generate(**kwargs)
File "/Users/sadesguy/Downloads/one-click-installers-main/oobabooga_macos/installer_files/env/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/Users/sadesguy/Downloads/one-click-installers-main/oobabooga_macos/installer_files/env/lib/python3.10/site-packages/transformers/generation/utils.py", line 1568, in generate
return self.sample(
File "/Users/sadesguy/Downloads/one-click-installers-main/oobabooga_macos/installer_files/env/lib/python3.10/site-packages/transformers/generation/utils.py", line 2615, in sample
outputs = self(
File "/Users/sadesguy/Downloads/one-click-installers-main/oobabooga_macos/installer_files/env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/sadesguy/Downloads/one-click-installers-main/oobabooga_macos/installer_files/env/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 945, in forward
outputs = self.model.decoder(
File "/Users/sadesguy/Downloads/one-click-installers-main/oobabooga_macos/installer_files/env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/sadesguy/Downloads/one-click-installers-main/oobabooga_macos/installer_files/env/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 655, in forward
pos_embeds = self.embed_positions(attention_mask, past_key_values_length)
File "/Users/sadesguy/Downloads/one-click-installers-main/oobabooga_macos/installer_files/env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/sadesguy/Downloads/one-click-installers-main/oobabooga_macos/installer_files/env/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 115, in forward
positions = (torch.cumsum(attention_mask, dim=1).type_as(attention_mask) * attention_mask).long() - 1
RuntimeError: MPS does not support cumsum op with int64 input
Output generated in 4.72 seconds (0.00 tokens/s, 0 tokens, context 63, seed 1888692510)
I am on the latest MacOs (13.4 (22f66)) on a MacBook air m2 16gb. I installed the opt6.7B version whilst choosing to run on CPU (also tried apple silicon, got same results). The issue occurs when I am in the localhost and when I press "Generate" with any prompt, I also enabled --mlock option, no other differences from default configuration.
What can cause this issue, and are there any fixes for it?
Traceback of the issue: