search

pytorch / xla

Enabling PyTorch on XLA Devices (e.g. Google TPU)
https://pytorch.org/xla
Other
2.48k stars 480 forks source link

Cannot Import _XLAC #7070

Open DarkenStar opened 5 months ago

DarkenStar commented 5 months ago

❓ Questions and Help

When I want to import torch_xla,the error occurs

>>> import torch_xla
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/code/pytorch/torch-xla/torch_xla/__init__.py", line 114, in <module>
    import _XLAC
ImportError: /code/pytorch/torch-xla/_XLAC.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZNK5torch8autograd4Node4nameEv

And I have followed the guide to make sure my torch version is the same as torch_xla https://github.com/Lightning-AI/pytorch-lightning/discussions/8320

>>> pip list | grep torch
[2]+  Stopped                 python
(torch_xla) root@0c9ffd606fd3:/code/pytorch/torch-xla# pip list | grep torch
rotary-embedding-torch    0.6.0
torch                     2.1.0+cu121        /root/miniconda3/envs/torch_xla/lib/python3.10/site-packages
torch-xla                 2.1.0              /code/pytorch/torch-xla
torchaudio                2.1.0+cu121
torchview                 0.2.6
torchvision               0.16.0+cu121
torchviz                  0.0.2

What should I do? TXS help

JackCaoG commented 5 months ago

@bhavya01 Can you take a look?

bhavya01 commented 5 months ago

@DarkenStar Can you provide more details about your environment or how you install the packages.

I did not see this in my setup:

sudo docker pull us-central1-docker.pkg.dev/tpu-pytorch-releases/docker/development:3.10_cuda_12.1
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit

# Configuring the NVIDIA Container Toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

sudo docker run --shm-size=16g --net=host --gpus all -it -d us-central1-docker.pkg.dev/tpu-pytorch-releases/docker/development:3.10_cuda_12.1
sudo docker exec -it $(sudo docker ps | awk 'NR==2 { print $1 }') /bin/bash
root@4a1ce18273e6:/# python
Python 3.10.14 (main, Apr 24 2024, 12:45:30) [GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> import torch_xla
>>> 
root@4a1ce18273e6:/#  pip list | grep torch
torch                    2.1.0
torch-xla                2.1.0
DarkenStar commented 5 months ago

@DarkenStar Can you provide more details about your environment or how you install the packages.

I did not see this in my setup:

sudo docker pull us-central1-docker.pkg.dev/tpu-pytorch-releases/docker/development:3.10_cuda_12.1
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit

# Configuring the NVIDIA Container Toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

sudo docker run --shm-size=16g --net=host --gpus all -it -d us-central1-docker.pkg.dev/tpu-pytorch-releases/docker/development:3.10_cuda_12.1
sudo docker exec -it $(sudo docker ps | awk 'NR==2 { print $1 }') /bin/bash
root@4a1ce18273e6:/# python
Python 3.10.14 (main, Apr 24 2024, 12:45:30) [GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> import torch_xla
>>> 
root@4a1ce18273e6:/#  pip list | grep torch
torch                    2.1.0
torch-xla                2.1.0

I first execute

git checkout 7bcf7da3a268b435777fe87c7794c382f444e86d
python setup.py develop

to compile pytorch 2.1.0, then clone xla code and execute

git checkout r2.1.0
XLA_CUDA=1 TF_CUDA_COMPUTE_CAPABILITIES=8.0 python setup.py develop

to compile xla, Thanks

DarkenStar commented 5 months ago

The command output is

Creating /root/miniconda3/envs/torch_xla/lib/python3.10/site-packages/torch-xla.egg-link (link to .)
torch-xla 2.1.0 is already the active version in easy-install.pth
Installing stablehlo-to-saved-model script to /root/miniconda3/envs/torch_xla/bin

Installed /code/pytorch/torch-xla
Processing dependencies for torch-xla==2.1.0
Searching for PyYAML==6.0.1
Best match: PyYAML 6.0.1
Adding PyYAML 6.0.1 to easy-install.pth file

Using /root/miniconda3/envs/torch_xla/lib/python3.10/site-packages
Searching for cloud-tpu-client==0.10
Best match: cloud-tpu-client 0.10
Processing cloud_tpu_client-0.10-py3.10.egg
cloud-tpu-client 0.10 is already the active version in easy-install.pth

Using /root/miniconda3/envs/torch_xla/lib/python3.10/site-packages/cloud_tpu_client-0.10-py3.10.egg
Searching for absl-py==2.0.0
Best match: absl-py 2.0.0
Processing absl_py-2.0.0-py3.10.egg
absl-py 2.0.0 is already the active version in easy-install.pth

Using /root/miniconda3/envs/torch_xla/lib/python3.10/site-packages/absl_py-2.0.0-py3.10.egg
Searching for oauth2client==4.1.3
Best match: oauth2client 4.1.3
Processing oauth2client-4.1.3-py3.10.egg
oauth2client 4.1.3 is already the active version in easy-install.pth

Using /root/miniconda3/envs/torch_xla/lib/python3.10/site-packages/oauth2client-4.1.3-py3.10.egg
Searching for google-api-python-client==1.8.0
Best match: google-api-python-client 1.8.0
Processing google_api_python_client-1.8.0-py3.10.egg
google-api-python-client 1.8.0 is already the active version in easy-install.pth

Using /root/miniconda3/envs/torch_xla/lib/python3.10/site-packages/google_api_python_client-1.8.0-py3.10.egg
Searching for six==1.16.0
Best match: six 1.16.0
Adding six 1.16.0 to easy-install.pth file

Using /root/miniconda3/envs/torch_xla/lib/python3.10/site-packages
Searching for rsa==4.9
Best match: rsa 4.9
Processing rsa-4.9-py3.10.egg
rsa 4.9 is already the active version in easy-install.pth
Installing pyrsa-decrypt script to /root/miniconda3/envs/torch_xla/bin
Installing pyrsa-encrypt script to /root/miniconda3/envs/torch_xla/bin
Installing pyrsa-keygen script to /root/miniconda3/envs/torch_xla/bin
Installing pyrsa-priv2pub script to /root/miniconda3/envs/torch_xla/bin
Installing pyrsa-sign script to /root/miniconda3/envs/torch_xla/bin
Installing pyrsa-verify script to /root/miniconda3/envs/torch_xla/bin

Using /root/miniconda3/envs/torch_xla/lib/python3.10/site-packages/rsa-4.9-py3.10.egg
Searching for pyasn1-modules==0.3.0
Best match: pyasn1-modules 0.3.0
Processing pyasn1_modules-0.3.0-py3.10.egg
pyasn1-modules 0.3.0 is already the active version in easy-install.pth

Using /root/miniconda3/envs/torch_xla/lib/python3.10/site-packages/pyasn1_modules-0.3.0-py3.10.egg
Searching for pyasn1==0.5.1
Best match: pyasn1 0.5.1
Processing pyasn1-0.5.1-py3.10.egg
pyasn1 0.5.1 is already the active version in easy-install.pth

Using /root/miniconda3/envs/torch_xla/lib/python3.10/site-packages/pyasn1-0.5.1-py3.10.egg
Searching for httplib2==0.22.0
Best match: httplib2 0.22.0
Processing httplib2-0.22.0-py3.10.egg
httplib2 0.22.0 is already the active version in easy-install.pth

Using /root/miniconda3/envs/torch_xla/lib/python3.10/site-packages/httplib2-0.22.0-py3.10.egg
Searching for uritemplate==3.0.1
Best match: uritemplate 3.0.1
Processing uritemplate-3.0.1-py3.10.egg
uritemplate 3.0.1 is already the active version in easy-install.pth

Using /root/miniconda3/envs/torch_xla/lib/python3.10/site-packages/uritemplate-3.0.1-py3.10.egg
Searching for google-api-core==1.34.0
Best match: google-api-core 1.34.0
Processing google_api_core-1.34.0-py3.10.egg
google-api-core 1.34.0 is already the active version in easy-install.pth

Using /root/miniconda3/envs/torch_xla/lib/python3.10/site-packages/google_api_core-1.34.0-py3.10.egg
Searching for google-auth-httplib2==0.1.1
Best match: google-auth-httplib2 0.1.1
Processing google_auth_httplib2-0.1.1-py3.10.egg
google-auth-httplib2 0.1.1 is already the active version in easy-install.pth

Using /root/miniconda3/envs/torch_xla/lib/python3.10/site-packages/google_auth_httplib2-0.1.1-py3.10.egg
Searching for google-auth==2.24.0
Best match: google-auth 2.24.0
Processing google_auth-2.24.0-py3.10.egg
google-auth 2.24.0 is already the active version in easy-install.pth

Using /root/miniconda3/envs/torch_xla/lib/python3.10/site-packages/google_auth-2.24.0-py3.10.egg
Searching for pyparsing==3.1.1
Best match: pyparsing 3.1.1
Processing pyparsing-3.1.1-py3.10.egg
pyparsing 3.1.1 is already the active version in easy-install.pth

Using /root/miniconda3/envs/torch_xla/lib/python3.10/site-packages/pyparsing-3.1.1-py3.10.egg
Searching for requests==2.31.0
Best match: requests 2.31.0
Adding requests 2.31.0 to easy-install.pth file

Using /root/miniconda3/envs/torch_xla/lib/python3.10/site-packages
Searching for protobuf==3.20.3
Best match: protobuf 3.20.3
Processing protobuf-3.20.3-py3.10.egg
protobuf 3.20.3 is already the active version in easy-install.pth

Using /root/miniconda3/envs/torch_xla/lib/python3.10/site-packages/protobuf-3.20.3-py3.10.egg
Searching for googleapis-common-protos==1.62.0rc1
Best match: googleapis-common-protos 1.62.0rc1
Processing googleapis_common_protos-1.62.0rc1-py3.10.egg
googleapis-common-protos 1.62.0rc1 is already the active version in easy-install.pth

Using /root/miniconda3/envs/torch_xla/lib/python3.10/site-packages/googleapis_common_protos-1.62.0rc1-py3.10.egg
Searching for cachetools==5.3.2
Best match: cachetools 5.3.2
Processing cachetools-5.3.2-py3.10.egg
cachetools 5.3.2 is already the active version in easy-install.pth

Using /root/miniconda3/envs/torch_xla/lib/python3.10/site-packages/cachetools-5.3.2-py3.10.egg
Searching for certifi==2024.2.2
Best match: certifi 2024.2.2
Adding certifi 2024.2.2 to easy-install.pth file

Using /root/miniconda3/envs/torch_xla/lib/python3.10/site-packages
Searching for urllib3==2.2.1
Best match: urllib3 2.2.1
Adding urllib3 2.2.1 to easy-install.pth file

Using /root/miniconda3/envs/torch_xla/lib/python3.10/site-packages
Searching for idna==3.7
Best match: idna 3.7
Adding idna 3.7 to easy-install.pth file

Using /root/miniconda3/envs/torch_xla/lib/python3.10/site-packages
Searching for charset-normalizer==3.3.2
Best match: charset-normalizer 3.3.2
Adding charset-normalizer 3.3.2 to easy-install.pth file
Installing normalizer script to /root/miniconda3/envs/torch_xla/bin

Using /root/miniconda3/envs/torch_xla/lib/python3.10/site-packages
Finished processing dependencies for torch-xla==2.1.0