search

TheStageAI / TorchIntegral

Integral Neural Networks in PyTorch
Apache License 2.0
122 stars 10 forks source link

Building module torch_integral.tsp_solver.solver failed #15

Closed guanyuelee closed 1 year ago

guanyuelee commented 1 year ago

Hi, I encountered an importing error when I am running the code in the Convert your model to integral models of the Readme file. 

import torch
import torch_integral as inn
from torchvision.models import resnet18

model = resnet18(pretrained=True)
wrapper = inn.IntegralWrapper(init_from_discrete=True)

# Specify continuous dimensions which you want to prune
continuous_dims = {
    "layer4.0.conv1.weight": [0],
    "layer4.1.conv1.weight": [0, 1]
}

# Convert to integral model
inn_model = wrapper(model, (1, 3, 224, 224), continuous_dims)

My environment is: 

# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
_openmp_mutex             5.1                       1_gnu    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
accelerate                0.20.3                   pypi_0    pypi
bzip2                     1.0.8                h7b6447c_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
ca-certificates           2023.05.30           h06a4308_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
catalyst                  22.4                     pypi_0    pypi
certifi                   2023.5.7                 pypi_0    pypi
charset-normalizer        3.2.0                    pypi_0    pypi
cmake                     3.26.4                   pypi_0    pypi
cython                    0.29.36                  pypi_0    pypi
filelock                  3.12.2                   pypi_0    pypi
hydra-slayer              0.4.1                    pypi_0    pypi
idna                      3.4                      pypi_0    pypi
jinja2                    3.1.2                    pypi_0    pypi
ld_impl_linux-64          2.38                 h1181459_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libffi                    3.4.4                h6a678d5_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libgcc-ng                 11.2.0               h1234567_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libgomp                   11.2.0               h1234567_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libstdcxx-ng              11.2.0               h1234567_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libuuid                   1.41.5               h5eee18b_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
lit                       16.0.6                   pypi_0    pypi
markupsafe                2.1.3                    pypi_0    pypi
mpmath                    1.3.0                    pypi_0    pypi
ncurses                   6.4                  h6a678d5_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
networkx                  3.1                      pypi_0    pypi
numpy                     1.25.1                   pypi_0    pypi
nvidia-cublas-cu11        11.10.3.66               pypi_0    pypi
nvidia-cuda-cupti-cu11    11.7.101                 pypi_0    pypi
nvidia-cuda-nvrtc-cu11    11.7.99                  pypi_0    pypi
nvidia-cuda-runtime-cu11  11.7.99                  pypi_0    pypi
nvidia-cudnn-cu11         8.5.0.96                 pypi_0    pypi
nvidia-cufft-cu11         10.9.0.58                pypi_0    pypi
nvidia-curand-cu11        10.2.10.91               pypi_0    pypi
nvidia-cusolver-cu11      11.4.0.1                 pypi_0    pypi
nvidia-cusparse-cu11      11.7.4.91                pypi_0    pypi
nvidia-nccl-cu11          2.14.3                   pypi_0    pypi
nvidia-nvtx-cu11          11.7.91                  pypi_0    pypi
openssl                   3.0.9                h7f8727e_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
packaging                 23.1                     pypi_0    pypi
pillow                    10.0.0                   pypi_0    pypi
pip                       23.1.2          py310h06a4308_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
protobuf                  4.23.4                   pypi_0    pypi
psutil                    5.9.5                    pypi_0    pypi
python                    3.10.12              h955ad1f_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pytorchcv                 0.0.67                   pypi_0    pypi
pyyaml                    6.0                      pypi_0    pypi
readline                  8.2                  h5eee18b_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
requests                  2.31.0                   pypi_0    pypi
scipy                     1.11.1                   pypi_0    pypi
setuptools                67.8.0          py310h06a4308_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
sqlite                    3.41.2               h5eee18b_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
sympy                     1.12                     pypi_0    pypi
tensorboardx              2.6.1                    pypi_0    pypi
tk                        8.6.12               h1ccaba5_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
torch                     2.0.1                    pypi_0    pypi
torchaudio                2.0.2                    pypi_0    pypi
torchvision               0.15.2                   pypi_0    pypi
tqdm                      4.65.0                   pypi_0    pypi
triton                    2.0.0                    pypi_0    pypi
typing-extensions         4.7.1                    pypi_0    pypi
tzdata                    2023c                h04d1e81_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
urllib3                   2.0.3                    pypi_0    pypi
wheel                     0.38.4          py310h06a4308_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
xz                        5.4.2                h5eee18b_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
zlib                      1.2.13               h5eee18b_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main

I failed with the error: 

ssh://gyli@10.135.78.31:22/home/gyli/anaconda3/envs/inn/bin/python -u /home/gyli/dreams/TorchIntegral/test.py
/home/gyli/anaconda3/envs/inn/lib/python3.10/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /home/gyli/dreams/TorchIntegral/torch_integral/tsp_solver/solver.pyx
  tree = Parsing.p_module(s, pxd, full_module_name)
/home/gyli/.pyxbld/temp.linux-x86_64-cpython-310/pyrex/torch_integral/tsp_solver/solver.c:752:10: fatal error: numpy/arrayobject.h: No such file or directory
 #include "numpy/arrayobject.h"
          ^~~~~~~~~~~~~~~~~~~~~
compilation terminated.
Traceback (most recent call last):
  File "/home/gyli/anaconda3/envs/inn/lib/python3.10/site-packages/setuptools/_distutils/unixccompiler.py", line 185, in _compile
    self.spawn(compiler_so + cc_args + [src, '-o', obj] + extra_postargs)
  File "/home/gyli/anaconda3/envs/inn/lib/python3.10/site-packages/setuptools/_distutils/ccompiler.py", line 1041, in spawn
    spawn(cmd, dry_run=self.dry_run, **kwargs)
  File "/home/gyli/anaconda3/envs/inn/lib/python3.10/site-packages/setuptools/_distutils/spawn.py", line 70, in spawn
    raise DistutilsExecError(
distutils.errors.DistutilsExecError: command '/usr/bin/gcc' failed with exit code 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/gyli/anaconda3/envs/inn/lib/python3.10/site-packages/pyximport/pyximport.py", line 214, in load_module
    so_path = build_module(module_name, pyxfilename, pyxbuild_dir,
  File "/home/gyli/anaconda3/envs/inn/lib/python3.10/site-packages/pyximport/pyximport.py", line 186, in build_module
    so_path = pyxbuild.pyx_to_dll(pyxfilename, extension_mod,
  File "/home/gyli/anaconda3/envs/inn/lib/python3.10/site-packages/pyximport/pyxbuild.py", line 102, in pyx_to_dll
    dist.run_commands()
  File "/home/gyli/anaconda3/envs/inn/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
    self.run_command(cmd)
  File "/home/gyli/anaconda3/envs/inn/lib/python3.10/site-packages/setuptools/dist.py", line 1244, in run_command
    super().run_command(command)
  File "/home/gyli/anaconda3/envs/inn/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/home/gyli/anaconda3/envs/inn/lib/python3.10/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
    _build_ext.build_ext.run(self)
  File "/home/gyli/anaconda3/envs/inn/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
    self.build_extensions()
  File "/home/gyli/anaconda3/envs/inn/lib/python3.10/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
    _build_ext.build_ext.build_extensions(self)
  File "/home/gyli/anaconda3/envs/inn/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
    self._build_extensions_serial()
  File "/home/gyli/anaconda3/envs/inn/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
    self.build_extension(ext)
  File "/home/gyli/anaconda3/envs/inn/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 548, in build_extension
    objects = self.compiler.compile(
  File "/home/gyli/anaconda3/envs/inn/lib/python3.10/site-packages/setuptools/_distutils/ccompiler.py", line 600, in compile
    self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
  File "/home/gyli/anaconda3/envs/inn/lib/python3.10/site-packages/setuptools/_distutils/unixccompiler.py", line 187, in _compile
    raise CompileError(msg)
distutils.errors.CompileError: command '/usr/bin/gcc' failed with exit code 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/gyli/dreams/TorchIntegral/test.py", line 4, in <module>
    import torch_integral as inn
  File "/home/gyli/dreams/TorchIntegral/torch_integral/__init__.py", line 1, in <module>
    from .model import IntegralWrapper
  File "/home/gyli/dreams/TorchIntegral/torch_integral/model.py", line 11, in <module>
    from .permutation import NOptPermutation
  File "/home/gyli/dreams/TorchIntegral/torch_integral/permutation.py", line 2, in <module>
    from .tsp_solver import two_opt_find_permutation
  File "/home/gyli/dreams/TorchIntegral/torch_integral/tsp_solver/__init__.py", line 1, in <module>
    from .solver import two_opt_find_permutation
  File "/home/gyli/anaconda3/envs/inn/lib/python3.10/site-packages/pyximport/pyximport.py", line 459, in load_module
    module = load_module(fullname, self.path,
  File "/home/gyli/anaconda3/envs/inn/lib/python3.10/site-packages/pyximport/pyximport.py", line 231, in load_module
    raise exc.with_traceback(tb)
  File "/home/gyli/anaconda3/envs/inn/lib/python3.10/site-packages/pyximport/pyximport.py", line 214, in load_module
    so_path = build_module(module_name, pyxfilename, pyxbuild_dir,
  File "/home/gyli/anaconda3/envs/inn/lib/python3.10/site-packages/pyximport/pyximport.py", line 186, in build_module
    so_path = pyxbuild.pyx_to_dll(pyxfilename, extension_mod,
  File "/home/gyli/anaconda3/envs/inn/lib/python3.10/site-packages/pyximport/pyxbuild.py", line 102, in pyx_to_dll
    dist.run_commands()
  File "/home/gyli/anaconda3/envs/inn/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
    self.run_command(cmd)
  File "/home/gyli/anaconda3/envs/inn/lib/python3.10/site-packages/setuptools/dist.py", line 1244, in run_command
    super().run_command(command)
  File "/home/gyli/anaconda3/envs/inn/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/home/gyli/anaconda3/envs/inn/lib/python3.10/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
    _build_ext.build_ext.run(self)
  File "/home/gyli/anaconda3/envs/inn/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
    self.build_extensions()
  File "/home/gyli/anaconda3/envs/inn/lib/python3.10/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
    _build_ext.build_ext.build_extensions(self)
  File "/home/gyli/anaconda3/envs/inn/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
    self._build_extensions_serial()
  File "/home/gyli/anaconda3/envs/inn/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
    self.build_extension(ext)
  File "/home/gyli/anaconda3/envs/inn/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 548, in build_extension
    objects = self.compiler.compile(
  File "/home/gyli/anaconda3/envs/inn/lib/python3.10/site-packages/setuptools/_distutils/ccompiler.py", line 600, in compile
    self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
  File "/home/gyli/anaconda3/envs/inn/lib/python3.10/site-packages/setuptools/_distutils/unixccompiler.py", line 187, in _compile
    raise CompileError(msg)
ImportError: Building module torch_integral.tsp_solver.solver failed: ["distutils.errors.CompileError: command '/usr/bin/gcc' failed with exit code 1\n"]

It seems that I cannot compile the Cython code with my gcc compiler. Do you have any ideas to resolve this issue? Many thanks.

ArnoldMSU commented 1 year ago

Hello @guanyuelee ! Checking your issue

guanyuelee commented 1 year ago

It seems that I didn't follow the installation guide in the readme. I didn't install the TorchIntegral into my environment. I have fixed it. Thanks!