search

pyg-team / pytorch_geometric

Graph Neural Network Library for PyTorch
https://pyg.org
MIT License
21.5k stars 3.69k forks source link

Library Not Loaded Error - macOS Big Sur #6530

Open br0kej opened 1 year ago

br0kej commented 1 year ago

🐛 Describe the bug

I am current encountering an error when trying to convert networks graphs to pytorch geometric data on macOS Big Sur 11.7 using Python 3.10.9. It looks like it's an issue with the MacOS build of pyg.

Code to Re-create Error

import networkx as nx
from torch_geometric.utils import from_networkx

G = nx.DiGraph()

from_networkx(G)

The Error

Traceback (most recent call last):
  File "/Users/br0kej/Codez/bin2ml/py/load_and_process.py", line 7, in <module>
    from torch_geometric.utils import from_networkx
  File "/Users/br0kej/Codez/bin2ml/venv/lib/python3.10/site-packages/torch_geometric/__init__.py", line 4, in <module>
    import torch_geometric.data
  File "/Users/br0kej/Codez/bin2ml/venv/lib/python3.10/site-packages/torch_geometric/data/__init__.py", line 7, in <module>
    from .lightning_datamodule import (
  File "/Users/br0kej/Codez/bin2ml/venv/lib/python3.10/site-packages/torch_geometric/data/lightning_datamodule.py", line 12, in <module>
    from torch_geometric.loader import (
  File "/Users/br0kej/Codez/bin2ml/venv/lib/python3.10/site-packages/torch_geometric/loader/__init__.py", line 4, in <module>
    from .neighbor_loader import NeighborLoader
  File "/Users/br0kej/Codez/bin2ml/venv/lib/python3.10/site-packages/torch_geometric/loader/neighbor_loader.py", line 6, in <module>
    from torch_geometric.loader.node_loader import NodeLoader
  File "/Users/br0kej/Codez/bin2ml/venv/lib/python3.10/site-packages/torch_geometric/loader/node_loader.py", line 20, in <module>
    from torch_geometric.sampler.base import (
  File "/Users/br0kej/Codez/bin2ml/venv/lib/python3.10/site-packages/torch_geometric/sampler/__init__.py", line 2, in <module>
    from .neighbor_sampler import NeighborSampler
  File "/Users/br0kej/Codez/bin2ml/venv/lib/python3.10/site-packages/torch_geometric/sampler/neighbor_sampler.py", line 22, in <module>
    import pyg_lib  # noqa
  File "/Users/br0kej/Codez/bin2ml/venv/lib/python3.10/site-packages/pyg_lib/__init__.py", line 38, in <module>
    load_library('libpyg')
  File "/Users/br0kej/Codez/bin2ml/venv/lib/python3.10/site-packages/pyg_lib/__init__.py", line 35, in load_library
    torch.ops.load_library(spec.origin)
  File "/Users/br0kej/Codez/bin2ml/venv/lib/python3.10/site-packages/torch/_ops.py", line 573, in load_library
    ctypes.CDLL(path)
  File "/usr/local/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/ctypes/__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: dlopen(/Users/br0kej/Codez/bin2ml/venv/lib/python3.10/site-packages/libpyg.so, 6): Library not loaded: /Users/runner/hostedtoolcache/Python/3.10.8/x64/lib/libpython3.10.dylib
  Referenced from: /Users/br0kej/Codez/bin2ml/venv/lib/python3.10/site-packages/libpyg.so
  Reason: image not found

Possible Cause

From having a bit of a google, there are several examples of other issues reporting the errors with GitHub Runners and CI/CD failures (Example) with very similar paths in the error trace. Whilst this deffo' isn't out problem here, the very end of the error Library not loaded: /Users/runner/hostedtoolcache/Python/3.10.8/x64/lib/libpython3.10.dylib suggests that an absolute path is being dropped in somewhere to the MacOS build and might be causing this failure?

Environment

rusty1s commented 1 year ago

What happens if you run pip uninstall pyg_lib?

br0kej commented 1 year ago

It works just fine!

vict0rsch commented 1 year ago

FYI, similar error 

$ python -c "import torch_geometric;print(torch_geometric.__version__)"

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/victor/.virtualenvs/phast/lib/python3.9/site-packages/torch_geometric/__init__.py", line 4, in <module>
    import torch_geometric.data
  File "/Users/victor/.virtualenvs/phast/lib/python3.9/site-packages/torch_geometric/data/__init__.py", line 7, in <module>
    from .lightning_datamodule import (
  File "/Users/victor/.virtualenvs/phast/lib/python3.9/site-packages/torch_geometric/data/lightning_datamodule.py", line 12, in <module>
    from torch_geometric.loader import (
  File "/Users/victor/.virtualenvs/phast/lib/python3.9/site-packages/torch_geometric/loader/__init__.py", line 4, in <module>
    from .neighbor_loader import NeighborLoader
  File "/Users/victor/.virtualenvs/phast/lib/python3.9/site-packages/torch_geometric/loader/neighbor_loader.py", line 6, in <module>
    from torch_geometric.loader.node_loader import NodeLoader
  File "/Users/victor/.virtualenvs/phast/lib/python3.9/site-packages/torch_geometric/loader/node_loader.py", line 20, in <module>
    from torch_geometric.sampler.base import (
  File "/Users/victor/.virtualenvs/phast/lib/python3.9/site-packages/torch_geometric/sampler/__init__.py", line 2, in <module>
    from .neighbor_sampler import NeighborSampler
  File "/Users/victor/.virtualenvs/phast/lib/python3.9/site-packages/torch_geometric/sampler/neighbor_sampler.py", line 22, in <module>
    import pyg_lib  # noqa
  File "/Users/victor/.virtualenvs/phast/lib/python3.9/site-packages/pyg_lib/__init__.py", line 38, in <module>
    load_library('libpyg')
  File "/Users/victor/.virtualenvs/phast/lib/python3.9/site-packages/pyg_lib/__init__.py", line 35, in load_library
    torch.ops.load_library(spec.origin)
  File "/Users/victor/.virtualenvs/phast/lib/python3.9/site-packages/torch/_ops.py", line 255, in load_library
    ctypes.CDLL(path)
  File "/Users/victor/.pyenv/versions/3.9.2/lib/python3.9/ctypes/__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: dlopen(/Users/victor/.virtualenvs/phast/lib/python3.9/site-packages/libpyg.so, 0x0006): Library not loaded: /usr/local/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/Python
  Referenced from: /Users/victor/.virtualenvs/phast/lib/python3.9/site-packages/libpyg.so
  Reason: tried: '/usr/local/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/Python' (no such file), '/Library/Frameworks/Python.framework/Versions/3.10/Python' (no such file), '/System/Library/Frameworks/Python.framework/Versions/3.10/Python' (no such file)

Similar fix uninstalling pyg_lib

Setup
  • macOS 12.1 (21C52)
  • Python 3.9.2 (virtualenv)
  • Install
    • pip install torch==1.12.1
pip install pyg_lib torch_scatter torch_sparse torch_cluster torch_spline_conv torch_geometric -f https://data.pyg.org/whl/torch-1.12.0+cpu.html
rusty1s commented 8 months ago

I hit the same issue today :( I tried many different things, but in all scenarios the final binary was searching for a Python folder in /Library/Frameworks/Python.framework/Versions. Installing it from the universal install at https://www.python.org/downloads/macos/ fixed it for me.