Undefined Symbol _ZN3c106detail19maybe_wrap_dim_slowEllb

[fratajcz]

Hi!

I wanted to use your library while building an adaptation of TOGL. I installed it with pip and everything checked out, but now that I try to import it:

from torch_persistent_homology.persistent_homology_cpu import compute_persistence_homology_batched_mt

I get the following error:

from torch_persistent_homology.persistent_homology_cpu import compute_persistence_homology_batched_mt
ImportError: /home/user/miniconda/envs/pyg/lib/python3.7/site-packages/torch_persistent_homology/persistent_homology_cpu.cpython-37m-x86_64-linux-gnu.so: undefined symbol: _ZN3c106detail19maybe_wrap_dim_slowEllb

I am not familiar enough with C++ to fix this issue, do you know where to start?

If you need more context, I run the code in a Docker container Ubuntu 20.04, Pytorch LTS 1.8.2 CPU version. I am trying to carve out a standalone implementation of the TopologyLayer.

[Pseudomanifold]

Hi!

This looks like a linker issue—potentially a mismatch between different libraries. I'd suggest two things:

1. Trying out installation with pip -v or poetry install -vvv (the problem should not be related to poetry vs pip, but better safe than sorry).
2. Since this is Docker environment, can you try installing everything without a Miniconda/Anaconda environment? IME, these envs sometimes cause issues when it comes to compiler selection, which might explain the linker issue.

Can you send us the output of ldd when applied to the problematic library, ldd /home/user/miniconda/envs/pyg/lib/python3.7/site-packages/torch_persistent_homology/persistent_homology_cpu.cpython-37m-x86_64-linux-gnu.so?

[fratajcz]

Hi! Thanks for your response. Here is the ouput from pip -v:

(pyg) user@aa0f3fc7c483:/app/speos$ pip install -v 'git+https://github.com/ExpectationMax/torch_persistent_homology.git'
Using pip 22.1.2 from /home/user/miniconda/envs/pyg/lib/python3.7/site-packages/pip (python 3.7)
Collecting git+https://github.com/ExpectationMax/torch_persistent_homology.git
  Cloning https://github.com/ExpectationMax/torch_persistent_homology.git to /tmp/pip-req-build-swuk2ir5
  Running command git version
  git version 2.25.1
  Running command git clone --filter=blob:none https://github.com/ExpectationMax/torch_persistent_homology.git /tmp/pip-req-build-swuk2ir5
  Cloning into '/tmp/pip-req-build-swuk2ir5'...
  Running command git rev-parse HEAD
  eea88014ba6acd72f5003680afd7a26b4057ce23
  Resolved https://github.com/ExpectationMax/torch_persistent_homology.git to commit eea88014ba6acd72f5003680afd7a26b4057ce23
  Running command pip subprocess to install build dependencies
  Collecting poetry-core>=1.0.0
    Using cached poetry_core-1.1.0-py3-none-any.whl (526 kB)
  Collecting setuptools
    Using cached setuptools-65.3.0-py3-none-any.whl (1.2 MB)
  Collecting torch
    Using cached torch-1.12.1-cp37-cp37m-manylinux1_x86_64.whl (776.3 MB)
  Collecting importlib-metadata>=1.7.0
    Using cached importlib_metadata-4.12.0-py3-none-any.whl (21 kB)
  Collecting typing-extensions
    Using cached typing_extensions-4.3.0-py3-none-any.whl (25 kB)
  Collecting zipp>=0.5
    Using cached zipp-3.8.1-py3-none-any.whl (5.6 kB)
  Installing collected packages: zipp, typing-extensions, setuptools, torch, importlib-metadata, poetry-core
  Successfully installed importlib-metadata-4.12.0 poetry-core-1.1.0 setuptools-65.3.0 torch-1.12.1 typing-extensions-4.3.0 zipp-3.8.1
  Installing build dependencies ... done
  Running command Getting requirements to build wheel
  Getting requirements to build wheel ... done
  Running command Preparing metadata (pyproject.toml)
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: torch<2.0.0,>=1.7.1 in /home/user/miniconda/envs/pyg/lib/python3.7/site-packages (from torch-persistent-homology==0.1.0) (1.11.0)
Requirement already satisfied: typing_extensions in /home/user/miniconda/envs/pyg/lib/python3.7/site-packages (from torch<2.0.0,>=1.7.1->torch-persistent-homology==0.1.0) (4.3.0)
Building wheels for collected packages: torch-persistent-homology
  Running command Building wheel for torch-persistent-homology (pyproject.toml)
  running build
  running build_py
  creating /tmp/pip-req-build-swuk2ir5/build
  creating /tmp/pip-req-build-swuk2ir5/build/lib.linux-x86_64-cpython-37
  creating /tmp/pip-req-build-swuk2ir5/build/lib.linux-x86_64-cpython-37/torch_persistent_homology
  copying torch_persistent_homology/__init__.py -> /tmp/pip-req-build-swuk2ir5/build/lib.linux-x86_64-cpython-37/torch_persistent_homology
  copying torch_persistent_homology/unionfind.hh -> /tmp/pip-req-build-swuk2ir5/build/lib.linux-x86_64-cpython-37/torch_persistent_homology
  copying torch_persistent_homology/perisistent_homology_cpu.cpp -> /tmp/pip-req-build-swuk2ir5/build/lib.linux-x86_64-cpython-37/torch_persistent_homology
  running build_ext
  building 'torch_persistent_homology.persistent_homology_cpu' extension
  creating /tmp/pip-req-build-swuk2ir5/build/temp.linux-x86_64-cpython-37
  creating /tmp/pip-req-build-swuk2ir5/build/temp.linux-x86_64-cpython-37/torch_persistent_homology
  gcc -pthread -B /home/user/miniconda/envs/pyg/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/tmp/pip-build-env-uk68o9kj/overlay/lib/python3.7/site-packages/torch/include -I/tmp/pip-build-env-uk68o9kj/overlay/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/tmp/pip-build-env-uk68o9kj/overlay/lib/python3.7/site-packages/torch/include/TH -I/tmp/pip-build-env-uk68o9kj/overlay/lib/python3.7/site-packages/torch/include/THC -I/home/user/miniconda/envs/pyg/include/python3.7m -c torch_persistent_homology/perisistent_homology_cpu.cpp -o /tmp/pip-req-build-swuk2ir5/build/temp.linux-x86_64-cpython-37/torch_persistent_homology/perisistent_homology_cpu.o -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -DTORCH_EXTENSION_NAME=persistent_homology_cpu -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
  cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
  g++ -pthread -B /home/user/miniconda/envs/pyg/compiler_compat -Wl,--sysroot=/ -pthread -shared -B /home/user/miniconda/envs/pyg/compiler_compat -L/home/user/miniconda/envs/pyg/lib -Wl,-rpath=/home/user/miniconda/envs/pyg/lib -Wl,--no-as-needed -Wl,--sysroot=/ /tmp/pip-req-build-swuk2ir5/build/temp.linux-x86_64-cpython-37/torch_persistent_homology/perisistent_homology_cpu.o -L/tmp/pip-build-env-uk68o9kj/overlay/lib/python3.7/site-packages/torch/lib -lc10 -ltorch -ltorch_cpu -ltorch_python -o /tmp/pip-req-build-swuk2ir5/build/lib.linux-x86_64-cpython-37/torch_persistent_homology/persistent_homology_cpu.cpython-37m-x86_64-linux-gnu.so -Wl,-rpath,/tmp/pip-build-env-uk68o9kj/overlay/lib/python3.7/site-packages/torch/lib
  /tmp/pip-build-env-uk68o9kj/overlay/lib/python3.7/site-packages/torch/utils/cpp_extension.py:411: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
    warnings.warn(msg.format('we could not find ninja.'))
  Building wheel for torch-persistent-homology (pyproject.toml) ... done
  Created wheel for torch-persistent-homology: filename=torch_persistent_homology-0.1.0-cp37-cp37m-manylinux_2_31_x86_64.whl size=3139555 sha256=565552a13dffd876eaf8b51a5e0a05a8a5003c222b33077e28b1a8b770a2c023
  Stored in directory: /tmp/pip-ephem-wheel-cache-0_vh01vj/wheels/8e/f2/1c/ae07e49afeaf53f90c0955e3e47d7e6d62da8c21f4a7f5a40f
Successfully built torch-persistent-homology
Installing collected packages: torch-persistent-homology
  Running command git rev-parse HEAD
  eea88014ba6acd72f5003680afd7a26b4057ce23
Successfully installed torch-persistent-homology-0.1.0

and the ldd output:

(pyg) user@aa0f3fc7c483:/app/speos$ ldd /home/user/miniconda/envs/pyg/lib/python3.7/site-packages/torch_persistent_homology/persistent_homology_cpu.cpython-37m-x86_64-linux-gnu.so
        libc10.so => not found
        libtorch.so => not found
        libtorch_cpu.so => not found
        libtorch_python.so => not found
        libstdc++.so.6 => /home/user/miniconda/envs/pyg/lib/libstdc++.so.6 (0x000000400188d000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x0000004001aa1000)
        libgcc_s.so.1 => /home/user/miniconda/envs/pyg/lib/libgcc_s.so.1 (0x0000004001bf2000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x0000004001c0c000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x0000004001c2f000)
        /lib64/ld-linux-x86-64.so.2 (0x0000004000000000)

libc10.so seems to be missing, but once I import torch it can be found (however, without the instruction that generates the error):

(pyg) user@aa0f3fc7c483:/app/speos$ python
Python 3.7.13 (default, Mar 29 2022, 02:18:16) 
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from torch_persistent_homology.persistent_homology_cpu import compute_persistence_homology_batched_mt
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: libc10.so: cannot open shared object file: No such file or directory
>>> import torch
>>> from torch_persistent_homology.persistent_homology_cpu import compute_persistence_homology_batched_mt
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: /home/user/miniconda/envs/pyg/lib/python3.7/site-packages/torch_persistent_homology/persistent_homology_cpu.cpython-37m-x86_64-linux-gnu.so: undefined symbol: _ZN3c106detail19maybe_wrap_dim_slowEllb
>>> exit()

Next I will install it without conda.

BTW I just found out that I can at least import the function on my mac in an anaconda env, so it really seems to be a platform issue. too bad macs don't come with CUDA support :/ Which OS did you run it on?

[fratajcz]

So, since libc10.so seems to be placed by pytorch, and i could import your module on mac using torch==1.12.0 but could not import you module on Ubuntu20.04 using torch==1.11.0, i went ahead and tried to import it on mac using torch==1.11.0:

$ pip install torch==1.11.0
$ pip -v install git+https://github.com/ExpectationMax/torch_persistent_homology.git
$ python
>>> import torch
>>> torch.__version__
'1.11.0'
>>> from torch_persistent_homology.persistent_homology_cpu import compute_persistence_homology_batched_mt
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: dlopen(/Users/florin.ratajczak/anaconda3/envs/homology2/lib/python3.7/site-packages/torch_persistent_homology/persistent_homology_cpu.cpython-37m-darwin.so, 2): Symbol not found: __ZN3c106detail19maybe_wrap_dim_slowExxb
  Referenced from: /Users/florin.ratajczak/anaconda3/envs/homology2/lib/python3.7/site-packages/torch_persistent_homology/persistent_homology_cpu.cpython-37m-darwin.so
  Expected in: /Users/florin.ratajczak/anaconda3/envs/homology2/lib/python3.7/site-packages/torch/lib/libc10.dylib
 in /Users/florin.ratajczak/anaconda3/envs/homology2/lib/python3.7/site-packages/torch_persistent_homology/persistent_homology_cpu.cpython-37m-darwin.so

So this might simply be a torch version issue with 1.11.0?

However, this time i got a warning during installation of torch_persistent_homology regarding the compiler:

                                 !! WARNING !!

  !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
  Your compiler (g++) is not compatible with the compiler Pytorch was
  built with for this platform, which is clang++ on darwin. Please
  use clang++ to to compile your extension. Alternatively, you may
  compile PyTorch from source using g++, and then you can also use
  g++ to compile your extension.

  See https://github.com/pytorch/pytorch/blob/master/CONTRIBUTING.md for help
  with compiling PyTorch from source.
  !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

                                !! WARNING !!

[Pseudomanifold]

Yes, this seems like a compiler problem—we definitely support both Linux and OS X (I have in fact used the package under both OSes). Can you try to set up the Docker without Conda but with Python 3.8? For now, this is the preferred version of the package.

I am confident that we can make this work!

[fratajcz]

Okay, using the following Dockerfile:

FROM --platform=linux/amd64 ubuntu:20.04

RUN apt-get update && apt-get install -y apt-transport-https ca-certificates && \
    rm -rf /var/lib/apt/lists/*

RUN apt-get update && apt-get install -y --no-install-recommends apt-utils gnupg2 curl build-essential git openssh-client wget python3.8 python3-pip python3-dev python3.8-dev libpython3.8-dev

# Create a working directory.
RUN mkdir app
WORKDIR app

# Create a non-root user and switch to it.
RUN adduser --disabled-password --gecos '' --shell /bin/bash user \
 && chown -R user:user /app
#RUN echo "user ALL=(ALL) NOPASSWD:ALL" > /etc/sudoers.d/90-user
USER user

# All users can use /home/user as their home directory.
ENV HOME=/home/user
RUN chmod 777 /home/user

RUN alias python='python3.8'

CMD ["/bin/bash"]

and installing poetry, adding ssh keys, cloning the github repo and running poetry install -vvv I was able to install torch_persistent_homology in python 3.8

Interestingly, I could not install it with pip in this container.

However, this now used torch==1.7.1 from the toml file, but I have to make it work in a conda managed env with torch==1.11.0

[Pseudomanifold]

Great! Can you try updating the torch version in the requirements.txt? There might be some small adjustments to do in the code, but I can support you in this.

[fratajcz]

So, I updated the corresponding line in the toml file from torch = "^1.7.1" to torch = "^1.11.0" but poetry was unable to locate this version. Actually theres some very long Issue threads on poetrys GitHub reporting various bugs when installing pytorch, so I went back to pip again.

I created a new env using poetry install -vvv, had it fail, activated it anyway, proceeded with pip install ., installation went through fine.

When importing torch_persistent_homology, the old error came up again, so it seems to be that the torch version causes it:

(torch-persistent-homology-py3.8) user@bdaabe9169f3:~$ python
Python 3.8.10 (default, Jun 22 2022, 20:18:18) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> from torch_persistent_homology.persistent_homology_cpu import compute_persistence_homology_batched_mt
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: /home/user/.cache/pypoetry/virtualenvs/torch-persistent-homology-dHzYfWS--py3.8/lib/python3.8/site-packages/torch_persistent_homology/persistent_homology_cpu.cpython-38-x86_64-linux-gnu.so: undefined symbol: _ZN3c106detail19maybe_wrap_dim_slowEllb

[Pseudomanifold]

Seems indeed like there's a mismatch between the torch versions. Can you try update to a different torch version and recompiling the torch_persistent_homology module?

[fratajcz]

Okay, it seems to get weirder. I created a new venv and installed the package with pip for each version.

working: 1.12.1+cu102

not working: 1.7.1, 1.8.1, 1.9.0:

File "<stdin>", line 1, in <module>
ImportError: /home/user/.cache/pypoetry/virtualenvs/torch182-yQ7Kl9Uc-py3.8/lib/python3.8/site-packages/torch_persistent_homology/persistent_homology_cpu.cpython-38-x86_64-linux-gnu.so: undefined symbol: _ZNK2at10TensorBase8data_ptrIdEEPT_v

1.10.0, 1.11.0:

  File "<stdin>", line 1, in <module>
ImportError: /home/user/.cache/pypoetry/virtualenvs/torch-persistent-homology-dHzYfWS--py3.8/lib/python3.8/site-packages/torch_persistent_homology/persistent_homology_cpu.cpython-38-x86_64-linux-gnu.so: undefined symbol: _ZN3c106detail19maybe_wrap_dim_slowEllb

interestingly it didnt work with the pip-installed 1.7.1 but it did work with the poetry-installed 1.7.1 (as mentioned earlier). So far it worked with 1.12.0 on mac and 1.12.1(likely also 1.12.0) on Ubuntu 20.04. I don't know enough about the interplay of poetry and pip (since poetry uses pip internally) to pinpoint why one works and the other does not for 1.7.1.

Sadly there is no torch_geometric for torch 1.12.x yet, so either we sort out the issue with 1.11.0 or we'll have to wait a little.

Edit: So far I get the feeling that maybe installing the other versions with poetry might do the trick. However, I keep encountering this error in poetry:

Because torch_persistent_homology depends on torch (1.11.0) which doesn't match any versions, version solving failed.

when I set torch = "^1.7.1" to torch = "^1.11.0" in the pyproject.toml file. However, it definitely exists.

Edit2: I just had to run poetry lock --no-update first, now it's installing, lets hope for the best...

Edit3: Nope, still the same error, even when torch 1.11.0 and torch_persistent_homology have been installed with poetry.

[Pseudomanifold]

What's the difference between the two environments? Do both of them have CUDA installed? One thing that I'd try here is running pip with --no-binary :all: flags. That way, it's ensured that everything is being built from scratch as opposed to using prebuilt packages.

[fratajcz]

I tried it with the --no-binary :all:flag in front and after the . of pip install ., with no success. It seems like torch is a special child. running just pip install . works, although then we have the Undefined Symbol error.

(torch111-py3.8) user@bdaabe9169f3:~/torch111$ pip install . --no-binary :all:
Processing /home/user/torch111
  Installing build dependencies ... error
  error: subprocess-exited-with-error

  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [16 lines of output]
      Collecting poetry-core>=1.0.0
        Using cached poetry-core-1.1.0.tar.gz (508 kB)
        Getting requirements to build wheel: started
        Getting requirements to build wheel: finished with status 'done'
        Preparing metadata (pyproject.toml): started
        Preparing metadata (pyproject.toml): finished with status 'done'
      Collecting setuptools
        Using cached setuptools-65.3.0.tar.gz (2.6 MB)
        Getting requirements to build wheel: started
        Getting requirements to build wheel: finished with status 'done'
        Installing backend dependencies: started
        Installing backend dependencies: finished with status 'done'
        Preparing metadata (pyproject.toml): started
        Preparing metadata (pyproject.toml): finished with status 'done'
      ERROR: Could not find a version that satisfies the requirement torch (from versions: none)
      ERROR: No matching distribution found for torch
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

(torch111-py3.8) user@bdaabe9169f3:~/torch111$ pip install .                  
Processing /home/user/torch111
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: torch==1.11.0 in /home/user/.cache/pypoetry/virtualenvs/torch111-kPxgPMdi-py3.8/lib/python3.8/site-packages (from torch-persistent-homology==0.1.0) (1.11.0)
Requirement already satisfied: typing-extensions in /home/user/.cache/pypoetry/virtualenvs/torch111-kPxgPMdi-py3.8/lib/python3.8/site-packages (from torch==1.11.0->torch-persistent-homology==0.1.0) (4.3.0)
Building wheels for collected packages: torch-persistent-homology
  Building wheel for torch-persistent-homology (pyproject.toml) ... done
  Created wheel for torch-persistent-homology: filename=torch_persistent_homology-0.1.0-cp38-cp38-manylinux_2_31_x86_64.whl size=3106135 sha256=0ab2f68ec5f727e7b660f72f2e2493e13832b5e4411b4c35f81d8449a1447100
  Stored in directory: /tmp/pip-ephem-wheel-cache-4n22qz75/wheels/2c/e4/b4/17a71bb95bda4404b3a2bba36a4b8f223c7b7e0fd25346b404
Successfully built torch-persistent-homology
Installing collected packages: torch-persistent-homology
  Attempting uninstall: torch-persistent-homology
    Found existing installation: torch-persistent-homology 0.1.0
    Not uninstalling torch-persistent-homology at /home/user/torch111, outside environment /home/user/.cache/pypoetry/virtualenvs/torch111-kPxgPMdi-py3.8
    Can't uninstall 'torch-persistent-homology'. No files were found to uninstall.
Successfully installed torch-persistent-homology-0.1.0
(torch111-py3.8) user@bdaabe9169f3:~/torch111$ cd ..
(torch111-py3.8) user@bdaabe9169f3:~$ python
Python 3.8.10 (default, Jun 22 2022, 20:18:18) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
/home/user/.cache/pypoetry/virtualenvs/torch111-kPxgPMdi-py3.8/lib/python3.8/site-packages/torch/_masked/__init__.py:223: UserWarning: Failed to initialize NumPy: numpy.core.multiarray failed to import (Triggered internally at  ../torch/csrc/utils/tensor_numpy.cpp:68.)
  example_input = torch.tensor([[-3, -2, -1], [0, 1, 2]])
>>> torch.__version__
'1.11.0+cu102'
>>> from torch_persistent_homology.persistent_homology_cpu import compute_persistence_homology_batched_mt
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: /home/user/.cache/pypoetry/virtualenvs/torch111-kPxgPMdi-py3.8/lib/python3.8/site-packages/torch_persistent_homology/persistent_homology_cpu.cpython-38-x86_64-linux-gnu.so: undefined symbol: _ZN3c106detail19maybe_wrap_dim_slowEllb
>>> exit()
(torch111-py3.8) user@bdaabe9169f3:~$ 

[Pseudomanifold]

Yes, this looks somewhat troublesome—is this with CUDA or without CUDA? Are we still in a Docker environment? In any case, I'd suggest removing this dependency from pyproject.toml and trying to install it manually. The output of the first command still leads me to believe that some version matching is going on.

[fratajcz]

We are still in the Docker container for which the Dockerfile is shown above. It doesn't have any CUDA drivers installed, but apparently torch installs the CUDA versions of their packages by default unless specified otherwise.

[Pseudomanifold]

Ah, maybe that's an issue. Can you try this type of installation:

pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cpu

[fratajcz]

Okay, so I removed torch and torch_persistent_homology, purged the pip cache and re-installed torch==1.11.0+cpuwith the line that I obtained from the official site:

pip install torch==1.11.0+cpu torchvision==0.12.0+cpu torchaudio==0.11.0 --extra-index-url https://download.pytorch.org/whl/cpu

Then, I tried to remove torch from the pyproject.toml dependencies, but then it wasn't importable during build:

pyproject.toml:

[tool.poetry]
name = "torch-persistent-homology"
version = "0.1.0"
description = ""
authors = []
build = 'build.py'

[tool.poetry.dependencies]
python = ">=3.7,<3.9"
torch = "^1.7.1"

[tool.poetry.dev-dependencies]
pytest = "^6.2.1"
pyper = {git = "git@github.com:Pseudomanifold/pyper.git"}

[build-system]
requires = ["poetry-core>=1.0.0", "setuptools"]
build-backend = "poetry.core.masonry.api"

Stack Trace:

(torch111-py3.8) user@bdaabe9169f3:~/torch111$ pip install -v .
Using pip 22.2.2 from /home/user/.cache/pypoetry/virtualenvs/torch111-kPxgPMdi-py3.8/lib/python3.8/site-packages/pip (python 3.8)
Processing /home/user/torch111
  Running command pip subprocess to install build dependencies
  Collecting poetry-core>=1.0.0
    Downloading poetry_core-1.1.0-py3-none-any.whl (526 kB)
       ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 526.2/526.2 kB 6.8 MB/s eta 0:00:00
  Collecting setuptools
    Downloading setuptools-65.3.0-py3-none-any.whl (1.2 MB)
       ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 9.5 MB/s eta 0:00:00
  Installing collected packages: setuptools, poetry-core
  Successfully installed poetry-core-1.1.0 setuptools-65.3.0
  Installing build dependencies ... done
  Running command Getting requirements to build wheel
  Getting requirements to build wheel ... done
  Running command Preparing metadata (pyproject.toml)
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: torch-persistent-homology
  Running command Building wheel for torch-persistent-homology (pyproject.toml)
  A setup.py file already exists. Using it.
  Traceback (most recent call last):
    File "/home/user/torch111/setup.py", line 24, in <module>
      from build import *
    File "/home/user/torch111/build.py", line 2, in <module>
      from torch.utils import cpp_extension
  ModuleNotFoundError: No module named 'torch'
  Traceback (most recent call last):
    File "/home/user/.cache/pypoetry/virtualenvs/torch111-kPxgPMdi-py3.8/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
      main()
    File "/home/user/.cache/pypoetry/virtualenvs/torch111-kPxgPMdi-py3.8/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/home/user/.cache/pypoetry/virtualenvs/torch111-kPxgPMdi-py3.8/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 261, in build_wheel
      return _build_backend().build_wheel(wheel_directory, config_settings,
    File "/tmp/pip-build-env-3x2ey21z/overlay/lib/python3.8/site-packages/poetry/core/masonry/api.py", line 67, in build_wheel
      return WheelBuilder.make_in(poetry, Path(wheel_directory))
    File "/tmp/pip-build-env-3x2ey21z/overlay/lib/python3.8/site-packages/poetry/core/masonry/builders/wheel.py", line 79, in make_in
      wb.build(target_dir=directory)
    File "/tmp/pip-build-env-3x2ey21z/overlay/lib/python3.8/site-packages/poetry/core/masonry/builders/wheel.py", line 113, in build
      self._build(zip_file)
    File "/tmp/pip-build-env-3x2ey21z/overlay/lib/python3.8/site-packages/poetry/core/masonry/builders/wheel.py", line 167, in _build
      self._run_build_command(setup)
    File "/tmp/pip-build-env-3x2ey21z/overlay/lib/python3.8/site-packages/poetry/core/masonry/builders/wheel.py", line 205, in _run_build_command
      subprocess.check_call(
    File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '['/home/user/.cache/pypoetry/virtualenvs/torch111-kPxgPMdi-py3.8/bin/python', '/home/user/torch111/setup.py', 'build', '-b', '/home/user/torch111/build']' returned non-zero exit status 1.
  error: subprocess-exited-with-error

  × Building wheel for torch-persistent-homology (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /home/user/.cache/pypoetry/virtualenvs/torch111-kPxgPMdi-py3.8/bin/python /home/user/.cache/pypoetry/virtualenvs/torch111-kPxgPMdi-py3.8/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /tmp/tmpmfe22pv9
  cwd: /home/user/torch111
  Building wheel for torch-persistent-homology (pyproject.toml) ... error
  ERROR: Failed building wheel for torch-persistent-homology
Failed to build torch-persistent-homology
ERROR: Could not build wheels for torch-persistent-homology, which is required to install pyproject.toml-based projects

So, adding torch back to the dependencies:

[tool.poetry]
name = "torch_persistent_homology"
version = "0.1.0"
description = ""
authors = []
build = 'build.py'

[tool.poetry.dependencies]
python = ">=3.7,<3.9"

[tool.poetry.dev-dependencies]
pytest = "^6.2.1"
pyper = {git = "git@github.com:Pseudomanifold/pyper.git"}

[build-system]
requires = ["poetry-core>=1.0.0", "setuptools", "torch"]
build-backend = "poetry.core.masonry.api"

it installed fine:

(torch111-py3.8) user@bdaabe9169f3:~/torch111$ pip install -v .
Using pip 22.2.2 from /home/user/.cache/pypoetry/virtualenvs/torch111-kPxgPMdi-py3.8/lib/python3.8/site-packages/pip (python 3.8)
Processing /home/user/torch111
  Running command pip subprocess to install build dependencies
  Collecting poetry-core>=1.0.0
    Using cached poetry_core-1.1.0-py3-none-any.whl (526 kB)
  Collecting setuptools
    Using cached setuptools-65.3.0-py3-none-any.whl (1.2 MB)
  Collecting torch
    Downloading torch-1.12.1-cp38-cp38-manylinux1_x86_64.whl (776.3 MB)
       ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 776.3/776.3 MB 2.1 MB/s eta 0:00:00
  Collecting typing-extensions
    Downloading typing_extensions-4.3.0-py3-none-any.whl (25 kB)
  Installing collected packages: typing-extensions, setuptools, poetry-core, torch
  ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
  torchvision 0.12.0+cpu requires torch==1.11.0, but you have torch 1.12.1 which is incompatible.
  torchaudio 0.11.0+cpu requires torch==1.11.0, but you have torch 1.12.1 which is incompatible.
  Successfully installed poetry-core-1.1.0 setuptools-65.3.0 torch-1.12.1 typing-extensions-4.3.0
  Installing build dependencies ... done
  Running command Getting requirements to build wheel
  Getting requirements to build wheel ... done
  Running command Preparing metadata (pyproject.toml)
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: torch-persistent-homology
  Running command Building wheel for torch-persistent-homology (pyproject.toml)
  A setup.py file already exists. Using it.
  running build
  running build_py
  running build_ext
  /tmp/pip-build-env-pzep2ip8/overlay/lib/python3.8/site-packages/torch/utils/cpp_extension.py:411: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
    warnings.warn(msg.format('we could not find ninja.'))
  Building wheel for torch-persistent-homology (pyproject.toml) ... done
  Created wheel for torch-persistent-homology: filename=torch_persistent_homology-0.1.0-cp38-cp38-manylinux_2_31_x86_64.whl size=3106118 sha256=103324c3436df7a8f3e823d661f25e98ecc401c16d926ae926a61c8edcadfd96
  Stored in directory: /tmp/pip-ephem-wheel-cache-2x8_3tqd/wheels/2c/e4/b4/17a71bb95bda4404b3a2bba36a4b8f223c7b7e0fd25346b404
Successfully built torch-persistent-homology
Installing collected packages: torch-persistent-homology
  Attempting uninstall: torch-persistent-homology
    Found existing installation: torch-persistent-homology 0.1.0
    Not uninstalling torch-persistent-homology at /home/user/torch111, outside environment /home/user/.cache/pypoetry/virtualenvs/torch111-kPxgPMdi-py3.8
    Can't uninstall 'torch-persistent-homology'. No files were found to uninstall.
Successfully installed torch-persistent-homology-0.1.0

It says it installed torch==1.12.1, but when I start python its still the previously installed 1.11.0+cpu, but with the same error as before:

 (torch111-py3.8) user@bdaabe9169f3:~/torch111$ python
Python 3.8.10 (default, Jun 22 2022, 20:18:18) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> torch.__version__
'1.11.0+cpu'
>>> from torch_persistent_homology.persistent_homology_cpu import compute_persistence_homology_batched_mt
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: /home/user/torch111/torch_persistent_homology/persistent_homology_cpu.cpython-38-x86_64-linux-gnu.so: undefined symbol: _ZN3c106detail19maybe_wrap_dim_slowEllb

However, the torch_geometric implementation for torch==1.12.* was just made available, so I will try to use 1.12.1 and see if it works.

[Pseudomanifold]

Hmm—can you try without specifying versions directly? (Also for pytorch-geometric: pip install torch-scatter torch-sparse torch-cluster torch-spline-conv torch-geometric -f https://data.pyg.org/whl/torch-1.11.0+cpu.html)

The fact that there is still a wrong version of torch around somewhere is troubling; this seems to indicate that there might be remnants of previous installations left.

[yingycl]

It seems that torch-persistent-homology only works for torch==1.12.1. Also maybe we need to degrade the scipy to use torch_geometric on torch==1.12.1. For me torch-persistent-homology, torch==1.12.1+cu113, torch-geometric==2.1.0, scipy==1.7.3 works simultaneouosly on Ubuntu 20.04.