Closed guangy10 closed 1 month ago
@msaroufim can you take a look?
@msaroufim Would these lines be the issue? Feels like something isn't imported correctly for OSS.
The problem seems to be that you're using torch cpu binaries with torchao gpu binaries, on pypi we default to cuda 12.1
Could you please try the below instead?
pip install torchao --extra-index-url https://download.pytorch.org/whl/cpu --force-reinstall
EDIT: I just tried this on my end and couldn't repro
(ao) [marksaroufim@devgpu003.cco3 ~]$ pip list | grep torch
pytorch-triton 3.0.0+dedb7bdf33
torch 2.5.0.dev20240716+cpu
torchao 0.3.1 /home/marksaroufim/anaconda3/envs/ao/lib/python3.10/site-packages
(ao) [marksaroufim@devgpu003.cco3 ~]$ python
Python 3.10.14 (main, May 6 2024, 19:42:50) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from torchao.kernel import intmm_triton
>>>
It's likely a linux specific issue because triton isn't installed or supported on mac so ensure that the version of pytorch is what you expect it to be and that triton is not installed. The specific error you're then hitting is probably due to having some old unsupported version of triton installed
The problem seems to be that you're using torch cpu binaries with torchao gpu binaries, on pypi we default to cuda 12.1
Could you please try the below instead?
pip install torchao --extra-index-url https://download.pytorch.org/whl/cpu --force-reinstall
EDIT: I just tried this on my end and couldn't repro
(ao) [marksaroufim@devgpu003.cco3 ~]$ pip list | grep torch pytorch-triton 3.0.0+dedb7bdf33 torch 2.5.0.dev20240716+cpu torchao 0.3.1 /home/marksaroufim/anaconda3/envs/ao/lib/python3.10/site-packages (ao) [marksaroufim@devgpu003.cco3 ~]$ python Python 3.10.14 (main, May 6 2024, 19:42:50) [GCC 11.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from torchao.kernel import intmm_triton >>>
It's likely a linux specific issue because triton isn't installed or supported on mac so ensure that the version of pytorch is what you expect it to be and that triton is not installed. The specific error you're then hitting is probably due to having some old unsupported version of triton installed
Yeah. Ok, so the triton is installed by the third-party/llava. And the installed version is 2.1.1, which is old. Let me try updating it to 3.0.0 instead.
Ok looks like this issue is unblocked, feel free to reopen if something else comes up
Thank you @msaroufim! the remaining issue in that PR is irrelevant. We can close this issue now.
Env:
In ExecuTorch https://github.com/pytorch/executorch/pull/4320, I ran into this error when trying to consolidate the setup for llava. The same setup works just fine on Mac. So it is a Linux specific import issue.
See the stacktrace: