search

pyg-team / pytorch_geometric

Graph Neural Network Library for PyTorch
https://pyg.org
MIT License
20.59k stars 3.57k forks source link

Installation with Poetry on Mac M1 ship #5947

Open robinbrochier opened 1 year ago

robinbrochier commented 1 year ago

😵 Describe the installation problem

I'm trying to install pytorch-geometric using a pyproject.toml file and the poetry install command. The configuration file is the following: 

[tool.poetry]
name = "my_project"
version = "0.1.0"
description = ""
authors = ["Me"]
readme = "README.md"

packages = [
    {include = "my_project"},
    {include = "scripts"}
]

[tool.poetry.dependencies]
python = "3.8.10"
torch ={url = "https://download.pytorch.org/whl/cpu/torch-1.12.1-cp38-none-macosx_11_0_arm64.whl"}
torchaudio = {url = "https://download.pytorch.org/whl/cpu/torchaudio-0.12.1-cp38-cp38-macosx_11_0_arm64.whl"}
torchvision = {url = "https://download.pytorch.org/whl/cpu/torchvision-0.13.1-cp38-cp38-macosx_11_0_arm64.whl"}
torch-scatter = {version = "2.0.9", source="pyg-cpu"}
torch-sparse = {version = "0.6.15", source="pyg-cpu",
torch-cluster = {version = "1.6.0", source="pyg-cpu"}
torch-spline-conv = {version = "1.2.1", source="pyg-cpu"}
torch-geometric = "2.1.0.post1"

[[tool.poetry.source]]
name = "pyg-cpu"
url = "https://data.pyg.org/whl/torch-1.12.0+cpu.html"
secondary = true

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

[mypy]
plugins = "numpy.typing.mypy_plugin"

But the dependencies for PyG can't be found:

Unable to find installation candidates for torch-spline-conv (1.2.1)

Do you know how to install PyG with Poetry ? I know there are no wheels for Mac M1 ship (at least before python 3.10), do you know how to make sure poetry builds it from source ?

Environment

robinbrochier commented 1 year ago

Just to add, one possibility is to use poethepoet by adding theses line in the toml file:

[tool.poe.tasks]
install-torch-cpu = "pip install torch==1.12.1 -f https://download.pytorch.org/whl/torch_stable.html"
install-pyg-cpu = "pip install torch-scatter torch-sparse torch-cluster torch-spline-conv torch-geometric"

then to run:

poetry install
poe install-torch-cpu
poe install-pyg-cpu

but the versions of torch libs won't be added to the package. It's not ideal.

marcosfelt commented 1 year ago

Thanks @robinbrochier! This worked with a minor change to install things from git:

install-pyg-cpu-git = "pip install git+https://github.com/pyg-team/pyg-lib.git git+https://github.com/rusty1s/pytorch_scatter.git git+https://github.com/rusty1s/pytorch_sparse.git git+https://github.com/rusty1s/pytorch_cluster.git git+https://github.com/rusty1s/pytorch_spline_conv.git"

Based on this thread.

svnv-svsv-jm commented 2 months ago

Has anyone solved this, in 2024?

I specify the source:

[[tool.poetry.source]]
name = "pyg-cpu"
url = "https://data.pyg.org/whl/torch-2.2.0+cpu.html"

and I still get 

Cannot install torch-sparse.

  - Installing torch-spline-conv (1.2.2+pt22cpu): Failed

  RuntimeError

  Unable to find installation candidates for torch-spline-conv (1.2.2+pt22cpu)

which is just crazy.

rusty1s commented 2 months ago 
[tool.poetry]
name = "test-package"
version = "0.1.0"
description = ""
authors = ["Matthias Fey <matthias@fey.com>"]

[tool.poetry.dependencies]
python = "3.10.14"
torch = {url = "https://download.pytorch.org/whl/cpu/torch-2.2.0%2Bcpu-cp310-cp310-linux_x86_64.whl"}
pyg_lib = {version="0.4.0", source = "pyg-cpu"}
torch_scatter = {version="2.1.2", source = "pyg-cpu"}
torch_spline_conv = {version="1.2.2", source = "pyg-cpu"}
torch_geometric = "2.5.3"

[[tool.poetry.source]]
name = "pypi_"
url = "https://pypi.org/simple/"

[[tool.poetry.source]]
name = "pyg-cpu"
url = "https://data.pyg.org/whl/torch-2.2.0+cpu.html"

works fine for me.

svnv-svsv-jm commented 2 months ago

works fine for me

You are on Linux...

torch = {url = "https://download.pytorch.org/whl/cpu/torch-2.2.0%2Bcpu-cp310-cp310-linux_x86_64.whl"}

I don't understand why you commented this? Can you elaborate?

svnv-svsv-jm commented 2 months ago

For me the problem seems to be that there are no wheels yet for ARM. I can install with pip if I first install torch. With poetry, poetry looks for the wheel, which does not exist, and it is unable to understand that it has to install torch first and then torch_scatter etc.

rusty1s commented 2 months ago

Yes, only pyg-lib nightly has ARM wheels right now, see https://data.pyg.org/whl/nightly/torch-2.2.0%2Bcpu.html.

thegodone commented 2 weeks ago

Do you plan to have ARM wheels for the others deps torch_scatter torch_spline_conv & torch-cluster ?

rusty1s commented 4 days ago

Yes, once PyTorch 2.4 is released.

thegodone commented 2 days ago

can we already play with RC version https://dev-discuss.pytorch.org/t/pytorch-release-2-4-0-final-rc-is-available/2213 ?

rusty1s commented 2 days ago

Yes. PyG itself is compatible with PyTorch 2.4. You can uninstall PyG extensions if you run into issues or install them from source if you need them.

thegodone commented 2 days ago

great my current stage after doing those installations:

conda create -n torch24 python==3.11
conda activate torch24
mamba install pytorch=2.4.0  -c pytorch-test
pip install numpy==1.26.4
pip install torch_geometric
pip install ninja

pip install --verbose git+https://github.com/pyg-team/pyg-lib.git

the error is:

Using pip 24.0 from /Users/tgg/miniforge3/envs/torch24/lib/python3.11/site-packages/pip (python 3.11)
Collecting git+https://github.com/pyg-team/pyg-lib.git
  Cloning https://github.com/pyg-team/pyg-lib.git to /private/var/folders/4w/xmf8nmhs51j4vjsttzcxmmm00000gn/T/pip-req-build-7epyq6do
  Running command git version
  git version 2.39.3 (Apple Git-146)
  Running command git clone --filter=blob:none https://github.com/pyg-team/pyg-lib.git /private/var/folders/4w/xmf8nmhs51j4vjsttzcxmmm00000gn/T/pip-req-build-7epyq6do
  Cloning into '/private/var/folders/4w/xmf8nmhs51j4vjsttzcxmmm00000gn/T/pip-req-build-7epyq6do'...
  Updating files:   1% (2/144)
  Updating files:   2% (3/144)
  Updating files:   3% (5/144)
  Updating files:   4% (6/144)
  Updating files:   5% (8/144)
  Updating files:   6% (9/144)
  Updating files:   7% (11/144)
  Updating files:   8% (12/144)
  Updating files:   9% (13/144)
  Updating files:  10% (15/144)
  Updating files:  11% (16/144)
  Updating files:  12% (18/144)
  Updating files:  13% (19/144)
  Updating files:  14% (21/144)
  Updating files:  15% (22/144)
  Updating files:  16% (24/144)
  Updating files:  17% (25/144)
  Updating files:  18% (26/144)
  Updating files:  19% (28/144)
  Updating files:  20% (29/144)
  Updating files:  21% (31/144)
  Updating files:  22% (32/144)
  Updating files:  23% (34/144)
  Updating files:  24% (35/144)
  Updating files:  25% (36/144)
  Updating files:  26% (38/144)
  Updating files:  27% (39/144)
  Updating files:  28% (41/144)
  Updating files:  29% (42/144)
  Updating files:  30% (44/144)
  Updating files:  31% (45/144)
  Updating files:  32% (47/144)
  Updating files:  33% (48/144)
  Updating files:  34% (49/144)
  Updating files:  35% (51/144)
  Updating files:  36% (52/144)
  Updating files:  37% (54/144)
  Updating files:  38% (55/144)
  Updating files:  39% (57/144)
  Updating files:  40% (58/144)
  Updating files:  41% (60/144)
  Updating files:  42% (61/144)
  Updating files:  43% (62/144)
  Updating files:  44% (64/144)
  Updating files:  45% (65/144)
  Updating files:  46% (67/144)
  Updating files:  47% (68/144)
  Updating files:  48% (70/144)
  Updating files:  49% (71/144)
  Updating files:  50% (72/144)
  Updating files:  51% (74/144)
  Updating files:  52% (75/144)
  Updating files:  53% (77/144)
  Updating files:  54% (78/144)
  Updating files:  55% (80/144)
  Updating files:  56% (81/144)
  Updating files:  57% (83/144)
  Updating files:  58% (84/144)
  Updating files:  59% (85/144)
  Updating files:  60% (87/144)
  Updating files:  61% (88/144)
  Updating files:  62% (90/144)
  Updating files:  63% (91/144)
  Updating files:  64% (93/144)
  Updating files:  65% (94/144)
  Updating files:  66% (96/144)
  Updating files:  67% (97/144)
  Updating files:  68% (98/144)
  Updating files:  69% (100/144)
  Updating files:  70% (101/144)
  Updating files:  71% (103/144)
  Updating files:  72% (104/144)
  Updating files:  73% (106/144)
  Updating files:  74% (107/144)
  Updating files:  75% (108/144)
  Updating files:  76% (110/144)
  Updating files:  77% (111/144)
  Updating files:  78% (113/144)
  Updating files:  79% (114/144)
  Updating files:  80% (116/144)
  Updating files:  81% (117/144)
  Updating files:  82% (119/144)
  Updating files:  83% (120/144)
  Updating files:  84% (121/144)
  Updating files:  85% (123/144)
  Updating files:  86% (124/144)
  Updating files:  87% (126/144)
  Updating files:  88% (127/144)
  Updating files:  89% (129/144)
  Updating files:  90% (130/144)
  Updating files:  91% (132/144)
  Updating files:  92% (133/144)
  Updating files:  93% (134/144)
  Updating files:  94% (136/144)
  Updating files:  95% (137/144)
  Updating files:  96% (139/144)
  Updating files:  97% (140/144)
  Updating files:  98% (142/144)
  Updating files:  99% (143/144)
  Updating files: 100% (144/144)
  Updating files: 100% (144/144), done.
  Running command git rev-parse HEAD
  c3355cb0b75d4a851bdfcd0d9ad20300fa6323ae
  Resolved https://github.com/pyg-team/pyg-lib.git to commit c3355cb0b75d4a851bdfcd0d9ad20300fa6323ae
  Running command git submodule update --init --recursive -q
  Running command git rev-parse HEAD
  c3355cb0b75d4a851bdfcd0d9ad20300fa6323ae
  Running command python setup.py egg_info
  running egg_info
  creating /private/var/folders/4w/xmf8nmhs51j4vjsttzcxmmm00000gn/T/pip-pip-egg-info-xzqffa18/pyg_lib.egg-info
  writing /private/var/folders/4w/xmf8nmhs51j4vjsttzcxmmm00000gn/T/pip-pip-egg-info-xzqffa18/pyg_lib.egg-info/PKG-INFO
  writing dependency_links to /private/var/folders/4w/xmf8nmhs51j4vjsttzcxmmm00000gn/T/pip-pip-egg-info-xzqffa18/pyg_lib.egg-info/dependency_links.txt
  writing requirements to /private/var/folders/4w/xmf8nmhs51j4vjsttzcxmmm00000gn/T/pip-pip-egg-info-xzqffa18/pyg_lib.egg-info/requires.txt
  writing top-level names to /private/var/folders/4w/xmf8nmhs51j4vjsttzcxmmm00000gn/T/pip-pip-egg-info-xzqffa18/pyg_lib.egg-info/top_level.txt
  writing manifest file '/private/var/folders/4w/xmf8nmhs51j4vjsttzcxmmm00000gn/T/pip-pip-egg-info-xzqffa18/pyg_lib.egg-info/SOURCES.txt'
  reading manifest file '/private/var/folders/4w/xmf8nmhs51j4vjsttzcxmmm00000gn/T/pip-pip-egg-info-xzqffa18/pyg_lib.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  warning: no previously-included files matching '*' found under directory 'docs'
  warning: no previously-included files matching '*' found under directory 'benchmark'
  adding license file 'LICENSE'
  adding license file 'LICENSE_radix_sort'
  writing manifest file '/private/var/folders/4w/xmf8nmhs51j4vjsttzcxmmm00000gn/T/pip-pip-egg-info-xzqffa18/pyg_lib.egg-info/SOURCES.txt'
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: pyg_lib
  Running command python setup.py bdist_wheel
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.macosx-11.0-arm64-cpython-311
  creating build/lib.macosx-11.0-arm64-cpython-311/pyg_lib
  copying pyg_lib/home.py -> build/lib.macosx-11.0-arm64-cpython-311/pyg_lib
  copying pyg_lib/__init__.py -> build/lib.macosx-11.0-arm64-cpython-311/pyg_lib
  copying pyg_lib/testing.py -> build/lib.macosx-11.0-arm64-cpython-311/pyg_lib
  copying pyg_lib/_triton.py -> build/lib.macosx-11.0-arm64-cpython-311/pyg_lib
  creating build/lib.macosx-11.0-arm64-cpython-311/pyg_lib/sampler
  copying pyg_lib/sampler/__init__.py -> build/lib.macosx-11.0-arm64-cpython-311/pyg_lib/sampler
  creating build/lib.macosx-11.0-arm64-cpython-311/pyg_lib/ops
  copying pyg_lib/ops/scatter_reduce.py -> build/lib.macosx-11.0-arm64-cpython-311/pyg_lib/ops
  copying pyg_lib/ops/__init__.py -> build/lib.macosx-11.0-arm64-cpython-311/pyg_lib/ops
  creating build/lib.macosx-11.0-arm64-cpython-311/pyg_lib/partition
  copying pyg_lib/partition/__init__.py -> build/lib.macosx-11.0-arm64-cpython-311/pyg_lib/partition
  running build_ext
  error: [Errno 2] No such file or directory: 'cmake'
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /Users/tgg/miniforge3/envs/torch24/bin/python3.11 -u -c '
  exec(compile('"'"''"'"''"'"'
  # This is <pip-setuptools-caller> -- a caller that pip uses to run setup.py
  #
  # - It imports setuptools before invoking setup.py, to enable projects that directly
  #   import from `distutils.core` to work with newer packaging standards.
  # - It provides a clear error message when setuptools is not installed.
  # - It sets `sys.argv[0]` to the underlying `setup.py`, when invoking `setup.py` so
  #   setuptools doesn'"'"'t think the script is `-c`. This avoids the following warning:
  #     manifest_maker: standard file '"'"'-c'"'"' not found".
  # - It generates a shim setup.py, for handling setup.cfg-only projects.
  import os, sys, tokenize

  try:
      import setuptools
  except ImportError as error:
      print(
          "ERROR: Can not execute `setup.py` since setuptools is not available in "
          "the build environment.",
          file=sys.stderr,
      )
      sys.exit(1)

  __file__ = %r
  sys.argv[0] = __file__

  if os.path.exists(__file__):
      filename = __file__
      with tokenize.open(__file__) as f:
          setup_py_code = f.read()
  else:
      filename = "<auto-generated setuptools caller>"
      setup_py_code = "from setuptools import setup; setup()"

  exec(compile(setup_py_code, filename, "exec"))
  '"'"''"'"''"'"' % ('"'"'/private/var/folders/4w/xmf8nmhs51j4vjsttzcxmmm00000gn/T/pip-req-build-7epyq6do/setup.py'"'"',), "<pip-setuptools-caller>", "exec"))' bdist_wheel -d /private/var/folders/4w/xmf8nmhs51j4vjsttzcxmmm00000gn/T/pip-wheel-0gnhqa6q
  cwd: /private/var/folders/4w/xmf8nmhs51j4vjsttzcxmmm00000gn/T/pip-req-build-7epyq6do/
  Building wheel for pyg_lib (setup.py) ... error
  ERROR: Failed building wheel for pyg_lib
  Running setup.py clean for pyg_lib
  Running command python setup.py clean
  running clean
  removing 'build/temp.macosx-11.0-arm64-cpython-311' (and everything under it)
  removing 'build/lib.macosx-11.0-arm64-cpython-311' (and everything under it)
  'build/bdist.macosx-11.0-arm64' does not exist -- can't clean it
  'build/scripts-3.11' does not exist -- can't clean it
  removing 'build'
Failed to build pyg_lib
ERROR: Could not build wheels for pyg_lib, which is required to install pyproject.toml-based projects

I am testing this code : https://gist.github.com/jgbrasier/69984b469bc774d2f8cd66181909061d#file-test_pyg-py

(torch24) tgg@arm64-apple-darwin20 infomaniak % python ex_pyg.py                                                 
/Users/tgg/miniforge3/envs/torch24/lib/python3.11/site-packages/torch_geometric/data/dataset.py:238: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  if osp.exists(f) and torch.load(f) != _repr(self.pre_transform):
/Users/tgg/miniforge3/envs/torch24/lib/python3.11/site-packages/torch_geometric/data/dataset.py:246: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  if osp.exists(f) and torch.load(f) != _repr(self.pre_filter):
/Users/tgg/miniforge3/envs/torch24/lib/python3.11/site-packages/torch_geometric/io/fs.py:215: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  return torch.load(f, map_location)
Cora dataset: Data(x=[2708, 1433], edge_index=[2, 10556], y=[2708], train_mask=[2708], val_mask=[2708], test_mask=[2708])
Traceback (most recent call last):
  File "/Users/tgg/Downloads/infomaniak/ex_pyg.py", line 13, in <module>
    n2v_model = Node2Vec(edge_index=data.edge_index, embedding_dim=128, walk_length=10, 
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/tgg/miniforge3/envs/torch24/lib/python3.11/site-packages/torch_geometric/nn/models/node2vec.py", line 67, in __init__
    raise ImportError(f"'{self.__class__.__name__}' "
ImportError: 'Node2Vec' requires either the 'pyg-lib' or 'torch-cluster' package
akihironitta commented 2 days ago

@thegodone Looks like you don't have cmake in your env:

error: [Errno 2] No such file or directory: 'cmake'
thegodone commented 2 days ago

of course I did not see the error:

conda create -n torch24 python==3.11
conda activate torch24
mamba install -y clang_osx-arm64 clangxx_osx-arm64 gfortran_osx-arm64 cmake 
mamba install -y pytorch=2.4.0  -c pytorch-test
pip install numpy==1.26.4
pip install torch_geometric
pip install ninja

MACOSX_DEPLOYMENT_TARGET=13.1 CC=clang CXX=clang++ pip install --verbose git+https://github.com/pyg-team/pyg-lib.git

Traceback (most recent call last): File "/Users/tgg/Downloads/infomaniak/ex_pyg.py", line 25, in for sample in loader: File "/Users/tgg/miniforge3/envs/torch24/lib/python3.11/site-packages/torch/utils/data/dataloader.py", line 440, in iter return self._get_iterator() ^^^^^^^^^^^^^^^^^^^^ File "/Users/tgg/miniforge3/envs/torch24/lib/python3.11/site-packages/torch/utils/data/dataloader.py", line 388, in _get_iterator return _MultiProcessingDataLoaderIter(self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/tgg/miniforge3/envs/torch24/lib/python3.11/site-packages/torch/utils/data/dataloader.py", line 1038, in init w.start() File "/Users/tgg/miniforge3/envs/torch24/lib/python3.11/multiprocessing/process.py", line 121, in start self._popen = self._Popen(self) ^^^^^^^^^^^^^^^^^ File "/Users/tgg/miniforge3/envs/torch24/lib/python3.11/multiprocessing/context.py", line 224, in _Popen return _default_context.get_context().Process._Popen(process_obj) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/tgg/miniforge3/envs/torch24/lib/python3.11/multiprocessing/context.py", line 288, in _Popen return Popen(process_obj) ^^^^^^^^^^^^^^^^^^ File "/Users/tgg/miniforge3/envs/torch24/lib/python3.11/multiprocessing/popen_spawn_posix.py", line 32, in init super().init(process_obj) File "/Users/tgg/miniforge3/envs/torch24/lib/python3.11/multiprocessing/popen_fork.py", line 19, in init self._launch(process_obj) File "/Users/tgg/miniforge3/envs/torch24/lib/python3.11/multiprocessing/popen_spawn_posix.py", line 47, in _launch reduction.dump(process_obj, fp) File "/Users/tgg/miniforge3/envs/torch24/lib/python3.11/multiprocessing/reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj) File "/Users/tgg/miniforge3/envs/torch24/lib/python3.11/site-packages/torch/multiprocessing/reductions.py", line 558, in reduce_storage metadata = storage._share_filenamecpu() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/tgg/miniforge3/envs/torch24/lib/python3.11/site-packages/torch/storage.py", line 337, in wrapper return fn(self, *args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/tgg/miniforge3/envs/torch24/lib/python3.11/site-packages/torch/storage.py", line 411, in _share_filenamecpu return super()._share_filenamecpu(args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ RuntimeError: _sharefilename: only available on CPU


so pyg did not support mps device. here the error with device : 'cpu'

File "/Users/tgg/Downloads/infomaniak/ex_pyg.py", line 25, in for sample in loader: File "/Users/tgg/miniforge3/envs/torch24/lib/python3.11/site-packages/torch/utils/data/dataloader.py", line 440, in iter return self._get_iterator() ^^^^^^^^^^^^^^^^^^^^ File "/Users/tgg/miniforge3/envs/torch24/lib/python3.11/site-packages/torch/utils/data/dataloader.py", line 388, in _get_iterator return _MultiProcessingDataLoaderIter(self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/tgg/miniforge3/envs/torch24/lib/python3.11/site-packages/torch/utils/data/dataloader.py", line 1038, in init w.start() File "/Users/tgg/miniforge3/envs/torch24/lib/python3.11/multiprocessing/process.py", line 121, in start self._popen = self._Popen(self) ^^^^^^^^^^^^^^^^^ File "/Users/tgg/miniforge3/envs/torch24/lib/python3.11/multiprocessing/context.py", line 224, in _Popen return _default_context.get_context().Process._Popen(process_obj) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/tgg/miniforge3/envs/torch24/lib/python3.11/multiprocessing/context.py", line 288, in _Popen return Popen(process_obj) ^^^^^^^^^^^^^^^^^^ File "/Users/tgg/miniforge3/envs/torch24/lib/python3.11/multiprocessing/popen_spawn_posix.py", line 32, in init super().init(process_obj) File "/Users/tgg/miniforge3/envs/torch24/lib/python3.11/multiprocessing/popen_fork.py", line 19, in init self._launch(process_obj) File "/Users/tgg/miniforge3/envs/torch24/lib/python3.11/multiprocessing/popen_spawn_posix.py", line 47, in _launch reduction.dump(process_obj, fp) File "/Users/tgg/miniforge3/envs/torch24/lib/python3.11/multiprocessing/reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj) TypeError: cannot pickle 'PyCapsule' object

rusty1s commented 2 days ago

Is this related to PyTorch 2.4? We haven't tested this one yet.

svnv-svsv-jm commented 1 day ago

This issue was about poetry, why are we discussing conda installs now? 😄