search

lab-cosmo / sphericart

Multi-language library for the calculation of spherical harmonics in Cartesian coordinates
https://sphericart.readthedocs.io/en/latest/
MIT License
73 stars 13 forks source link

Cannot install for JAX #124

Closed Chutlhu closed 5 months ago

Chutlhu commented 5 months ago

Dear all, great project! I look forward to using this in Jax, especially since the Jax.scipy implementation does not work well. I am trying to pip-install it, but I got the following error

WARNING: sphericart 0.3.0 does not provide the extra 'jax'

Then I tried installing from the cloned repository, but I get this error

         raise CalledProcessError(retcode, process.args,
      subprocess.CalledProcessError: Command '['cmake', '--build', '/XXXXXX/XXXXX/third_party/sphericart/sphericart-jax/build/cmake-build', '--parallel', '--target', 'install']' returned non-zero exit status 2.

(where XXXXX is my folder path) I also tried following this instruction, but the cmake compilation failed.

Any clue?

System info:

Luthaf commented 5 months ago

It seems that the jax version is still a beta on PyPI (@frostedoyster should we release 0.4.0 and get moving?), you should be able to install it with pip install --pre sphericart[jax]. If the CMake compilation fails again, could you share the corresponding output?

Chutlhu commented 5 months ago

Thank for the prompt answer. Adding --pre makes it point to the correct release, however the compilation still fails: here is the output:

pip install --pre "sphericart[jax]"
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting sphericart[jax]
  Downloading sphericart-0.4.0b1-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.7 kB)
Requirement already satisfied: numpy in /home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/site-packages (from sphericart[jax]) (1.26.4)
Collecting sphericart-jax (from sphericart[jax])
  Downloading sphericart-jax-0.4.0b1.tar.gz (46 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 46.3/46.3 kB 6.4 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: jax>=0.4.18 in /home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/site-packages (from sphericart-jax->sphericart[jax]) (0.4.29)
Requirement already satisfied: ml-dtypes>=0.4.0 in /home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/site-packages (from jax>=0.4.18->sphericart-jax->sphericart[jax]) (0.4.0)
Requirement already satisfied: opt-einsum in /home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/site-packages (from jax>=0.4.18->sphericart-jax->sphericart[jax]) (3.3.0)
Requirement already satisfied: scipy>=1.9 in /home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/site-packages (from jax>=0.4.18->sphericart-jax->sphericart[jax]) (1.13.1)
Requirement already satisfied: importlib-metadata>=4.6 in /home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/site-packages (from jax>=0.4.18->sphericart-jax->sphericart[jax]) (7.1.0)
Requirement already satisfied: zipp>=0.5 in /home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/site-packages (from importlib-metadata>=4.6->jax>=0.4.18->sphericart-jax->sphericart[jax]) (3.19.2)
Downloading sphericart-0.4.0b1-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.0 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.0/1.0 MB 10.6 MB/s eta 0:00:00
Building wheels for collected packages: sphericart-jax
  Building wheel for sphericart-jax (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for sphericart-jax (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [142 lines of output]
      /tmp/pip-build-env-3saom955/overlay/lib/python3.9/site-packages/setuptools/dist.py:320: InformationOnly: Normalizing '0.4.0-beta1' to '0.4.0b1'
        self.metadata.version = self._normalize_version(self.metadata.version)
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-39
      creating build/lib.linux-x86_64-cpython-39/sphericart
      creating build/lib.linux-x86_64-cpython-39/sphericart/jax
      copying python/sphericart/jax/spherical_harmonics.py -> build/lib.linux-x86_64-cpython-39/sphericart/jax
      copying python/sphericart/jax/__init__.py -> build/lib.linux-x86_64-cpython-39/sphericart/jax
      copying python/sphericart/jax/sph.py -> build/lib.linux-x86_64-cpython-39/sphericart/jax
      copying python/sphericart/jax/utils.py -> build/lib.linux-x86_64-cpython-39/sphericart/jax
      copying python/sphericart/jax/dsph.py -> build/lib.linux-x86_64-cpython-39/sphericart/jax
      copying python/sphericart/jax/ddsph.py -> build/lib.linux-x86_64-cpython-39/sphericart/jax
      running egg_info
      writing python/sphericart_jax.egg-info/PKG-INFO
      writing dependency_links to python/sphericart_jax.egg-info/dependency_links.txt
      writing requirements to python/sphericart_jax.egg-info/requires.txt
      writing top-level names to python/sphericart_jax.egg-info/top_level.txt
      reading manifest file 'python/sphericart_jax.egg-info/SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      writing manifest file 'python/sphericart_jax.egg-info/SOURCES.txt'
      running build_ext
      -- The CXX compiler identification is GNU 9.4.0
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /usr/bin/c++ - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- Looking for a CUDA compiler
      -- Looking for a CUDA compiler - /usr/bin/nvcc
      -- The CUDA compiler identification is NVIDIA 10.1.243
      -- Detecting CUDA compiler ABI info
      -- Detecting CUDA compiler ABI info - done
      -- Check for working CUDA compiler: /usr/bin/nvcc - skipped
      -- Detecting CUDA compile features
      -- Detecting CUDA compile features - done
      -- Setting build type to 'relwithdebinfo' as none was specified.
      CMake Warning (dev) at /tmp/pip-build-env-3saom955/overlay/lib/python3.9/site-packages/pybind11/share/cmake/pybind11/FindPythonLibsNew.cmake:101 (message):
        Policy CMP0148 is not set: The FindPythonInterp and FindPythonLibs modules
        are removed.  Run "cmake --help-policy CMP0148" for policy details.  Use
        the cmake_policy command to set the policy and suppress this warning, or
        preferably upgrade to using FindPython, either by calling it explicitly
        before pybind11, or by setting PYBIND11_FINDPYTHON ON before pybind11.
      Call Stack (most recent call first):
        /tmp/pip-build-env-3saom955/overlay/lib/python3.9/site-packages/pybind11/share/cmake/pybind11/pybind11Tools.cmake:50 (find_package)
        /tmp/pip-build-env-3saom955/overlay/lib/python3.9/site-packages/pybind11/share/cmake/pybind11/pybind11Common.cmake:202 (include)
        /tmp/pip-build-env-3saom955/overlay/lib/python3.9/site-packages/pybind11/share/cmake/pybind11/pybind11Config.cmake:250 (include)
        CMakeLists.txt:34 (find_package)
      This warning is for project developers.  Use -Wno-dev to suppress it.

      -- Found PythonInterp: /home/dicarlod/anaconda3/envs/python39cuda12/bin/python (found suitable version "3.9.19", minimum required is "3.7")
      -- Found PythonLibs: /home/dicarlod/anaconda3/envs/python39cuda12/lib/libpython3.9.so
      -- Performing Test HAS_FLTO
      -- Performing Test HAS_FLTO - Success
      -- Found pybind11: /tmp/pip-build-env-3saom955/overlay/lib/python3.9/site-packages/pybind11/include (found version "2.13.0")
      -- The C compiler identification is GNU 9.4.0
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: /usr/bin/cc - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY
      -- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY - Success
      -- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY
      -- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY - Success
      -- Performing Test COMPILER_HAS_DEPRECATED_ATTR
      -- Performing Test COMPILER_HAS_DEPRECATED_ATTR - Success
      -- Performing Test COMPILER_SUPPORTS_WPRAGMAS
      -- Performing Test COMPILER_SUPPORTS_WPRAGMAS - Success
      -- Found OpenMP_C: -fopenmp (found version "4.5")
      -- Found OpenMP_CXX: -fopenmp (found version "4.5")
      -- Found OpenMP: TRUE (found version "4.5")
      -- OpenMP is enabled
      -- Performing Test COMPILER_SUPPORTS_MARCH_NATIVE
      -- Performing Test COMPILER_SUPPORTS_MARCH_NATIVE - Success
      -- march=native is enabled
      -- Performing Test COMPILER_SUPPORTS_WALL
      -- Performing Test COMPILER_SUPPORTS_WALL - Success
      -- Performing Test COMPILER_SUPPORTS_WEXTRA
      -- Performing Test COMPILER_SUPPORTS_WEXTRA - Success
      -- Performing Test COMPILER_SUPPORTS_WDOUBLE_PROMOTION
      -- Performing Test COMPILER_SUPPORTS_WDOUBLE_PROMOTION - Success
      -- Performing Test COMPILER_SUPPORTS_WFLOAT_CONVERSION
      -- Performing Test COMPILER_SUPPORTS_WFLOAT_CONVERSION - Success
      -- Configuring done (6.1s)
      -- Generating done (0.0s)
      -- Build files have been written to: /tmp/pip-install-4_dpkefs/sphericart-jax_bae28d6715a545a09acbfe7d917d8dfb/build/cmake-build
      [ 10%] Building CXX object sphericart/CMakeFiles/sphericart.dir/src/sphericart.cpp.o
      [ 20%] Building CXX object sphericart/CMakeFiles/sphericart.dir/src/sphericart-capi.cpp.o
      [ 30%] Building CUDA object sphericart/CMakeFiles/sphericart.dir/src/cuda_base.cu.o
      nvcc fatal   : Unsupported gpu architecture 'compute_86'
      make[2]: *** [sphericart/CMakeFiles/sphericart.dir/build.make:104: sphericart/CMakeFiles/sphericart.dir/src/cuda_base.cu.o] Error 1
      make[1]: *** [CMakeFiles/Makefile2:153: sphericart/CMakeFiles/sphericart.dir/all] Error 2
      make: *** [Makefile:136: all] Error 2
      Traceback (most recent call last):
        File "/home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
          return _build_backend().build_wheel(wheel_directory, config_settings,
        File "/tmp/pip-build-env-3saom955/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 415, in build_wheel
          return self._build_with_temp_dir(
        File "/tmp/pip-build-env-3saom955/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 397, in _build_with_temp_dir
          self.run_setup()
        File "/tmp/pip-build-env-3saom955/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 313, in run_setup
          exec(code, locals())
        File "<string>", line 78, in <module>
        File "/tmp/pip-build-env-3saom955/overlay/lib/python3.9/site-packages/setuptools/__init__.py", line 103, in setup
          return distutils.core.setup(**attrs)
        File "/tmp/pip-build-env-3saom955/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 184, in setup
          return run_commands(dist)
        File "/tmp/pip-build-env-3saom955/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 200, in run_commands
          dist.run_commands()
        File "/tmp/pip-build-env-3saom955/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/tmp/pip-build-env-3saom955/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 976, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-3saom955/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-3saom955/overlay/lib/python3.9/site-packages/setuptools/command/bdist_wheel.py", line 373, in run
          self.run_command("build")
        File "/tmp/pip-build-env-3saom955/overlay/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
          self.distribution.run_command(command)
        File "/tmp/pip-build-env-3saom955/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 976, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-3saom955/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-3saom955/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build.py", line 132, in run
          self.run_command(cmd_name)
        File "/tmp/pip-build-env-3saom955/overlay/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
          self.distribution.run_command(command)
        File "/tmp/pip-build-env-3saom955/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 976, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-3saom955/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "<string>", line 55, in run
        File "/home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/subprocess.py", line 528, in run
          raise CalledProcessError(retcode, process.args,
      subprocess.CalledProcessError: Command '['cmake', '--build', '/tmp/pip-install-4_dpkefs/sphericart-jax_bae28d6715a545a09acbfe7d917d8dfb/build/cmake-build', '--target', 'install']' returned non-zero exit status 2.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for sphericart-jax
Failed to build sphericart-jax
ERROR: Could not build wheels for sphericart-jax, which is required to install pyproject.toml-based projects
nickjbrowning commented 5 months ago

Could you try modifying this line locally from "all" to "native" and let us know if that works?

https://github.com/lab-cosmo/sphericart/blob/5f162053a1aadefec79709103840fd4e6b27cdea/sphericart/CMakeLists.txt#L70

additionally please provide details on:

the output of nvidia-smi and nvcc --version

Chutlhu commented 5 months ago

Unfortunately, it didn't work. I modified the file, and I tried to compile it locally by cloning the repo and executing:

$ cd sphericart
$pip install .[jax]

here is the output

Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Processing /media/dicarlod/SSD_2/diego/Code/NeuralSteerer/nsteerer/third_party/sphericart
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: numpy in /home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/site-packages (from sphericart==0.4.0b1) (1.26.4)
Processing ./sphericart-jax (from sphericart==0.4.0b1)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: jax>=0.4.18 in /home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/site-packages (from sphericart-jax@ file:///media/dicarlod/SSD_2/diego/Code/NeuralSteerer/nsteerer/third_party/sphericart/sphericart-jax?3170f822-955e-4586-8d92-1c64277eed83->sphericart==0.4.0b1) (0.4.29)
Requirement already satisfied: ml-dtypes>=0.4.0 in /home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/site-packages (from jax>=0.4.18->sphericart-jax@ file:///media/dicarlod/SSD_2/diego/Code/NeuralSteerer/nsteerer/third_party/sphericart/sphericart-jax?3170f822-955e-4586-8d92-1c64277eed83->sphericart==0.4.0b1) (0.4.0)
Requirement already satisfied: opt-einsum in /home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/site-packages (from jax>=0.4.18->sphericart-jax@ file:///media/dicarlod/SSD_2/diego/Code/NeuralSteerer/nsteerer/third_party/sphericart/sphericart-jax?3170f822-955e-4586-8d92-1c64277eed83->sphericart==0.4.0b1) (3.3.0)
Requirement already satisfied: scipy>=1.9 in /home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/site-packages (from jax>=0.4.18->sphericart-jax@ file:///media/dicarlod/SSD_2/diego/Code/NeuralSteerer/nsteerer/third_party/sphericart/sphericart-jax?3170f822-955e-4586-8d92-1c64277eed83->sphericart==0.4.0b1) (1.13.1)
Requirement already satisfied: importlib-metadata>=4.6 in /home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/site-packages (from jax>=0.4.18->sphericart-jax@ file:///media/dicarlod/SSD_2/diego/Code/NeuralSteerer/nsteerer/third_party/sphericart/sphericart-jax?3170f822-955e-4586-8d92-1c64277eed83->sphericart==0.4.0b1) (7.1.0)
Requirement already satisfied: zipp>=0.5 in /home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/site-packages (from importlib-metadata>=4.6->jax>=0.4.18->sphericart-jax@ file:///media/dicarlod/SSD_2/diego/Code/NeuralSteerer/nsteerer/third_party/sphericart/sphericart-jax?3170f822-955e-4586-8d92-1c64277eed83->sphericart==0.4.0b1) (3.19.2)
Building wheels for collected packages: sphericart, sphericart-jax
  Building wheel for sphericart (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for sphericart (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [82 lines of output]
      /tmp/pip-build-env-t5pmetzk/overlay/lib/python3.9/site-packages/setuptools/dist.py:320: InformationOnly: Normalizing '0.4.0-beta1' to '0.4.0b1'
        self.metadata.version = self._normalize_version(self.metadata.version)
      running bdist_wheel
      running build
      running build_py
      copying python/src/sphericart/_c_lib.py -> build/lib.linux-x86_64-cpython-39/sphericart
      copying python/src/sphericart/__init__.py -> build/lib.linux-x86_64-cpython-39/sphericart
      copying python/src/sphericart/spherical_harmonics.py -> build/lib.linux-x86_64-cpython-39/sphericart
      running egg_info
      writing python/src/sphericart.egg-info/PKG-INFO
      writing dependency_links to python/src/sphericart.egg-info/dependency_links.txt
      writing requirements to python/src/sphericart.egg-info/requires.txt
      writing top-level names to python/src/sphericart.egg-info/top_level.txt
      reading manifest file 'python/src/sphericart.egg-info/SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      warning: no files found matching 'LICENSE'
      adding license file 'LICENSE-APACHE'
      adding license file 'LICENSE-MIT'
      adding license file 'LICENSE.md'
      writing manifest file 'python/src/sphericart.egg-info/SOURCES.txt'
      running build_ext
      -- OpenMP is enabled
      -- march=native is enabled
      -- Configuring done (0.0s)
      -- Generating done (0.0s)
      -- Build files have been written to: /media/dicarlod/SSD_2/diego/Code/NeuralSteerer/nsteerer/third_party/sphericart/build/cmake-build
      [ 20%] Building CUDA object CMakeFiles/sphericart.dir/src/cuda_base.cu.o
      [ 40%] Building CUDA object CMakeFiles/sphericart.dir/src/sphericart_cuda.cu.o
      nvcc fatal   : Unsupported gpu architecture 'compute_86'
      nvcc fatal   : Unsupported gpu architecture 'compute_86'
      make[2]: *** [CMakeFiles/sphericart.dir/build.make:104: CMakeFiles/sphericart.dir/src/cuda_base.cu.o] Error 1
      make[2]: *** Waiting for unfinished jobs....
      make[2]: *** [CMakeFiles/sphericart.dir/build.make:119: CMakeFiles/sphericart.dir/src/sphericart_cuda.cu.o] Error 1
      make[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/sphericart.dir/all] Error 2
      make: *** [Makefile:146: all] Error 2
      sphericart: CUDA_HOME:  None
      Traceback (most recent call last):
        File "/home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
          return _build_backend().build_wheel(wheel_directory, config_settings,
        File "/tmp/pip-build-env-t5pmetzk/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 415, in build_wheel
          return self._build_with_temp_dir(
        File "/tmp/pip-build-env-t5pmetzk/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 397, in _build_with_temp_dir
          self.run_setup()
        File "/tmp/pip-build-env-t5pmetzk/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 313, in run_setup
          exec(code, locals())
        File "<string>", line 120, in <module>
        File "/tmp/pip-build-env-t5pmetzk/overlay/lib/python3.9/site-packages/setuptools/__init__.py", line 103, in setup
          return distutils.core.setup(**attrs)
        File "/tmp/pip-build-env-t5pmetzk/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 184, in setup
          return run_commands(dist)
        File "/tmp/pip-build-env-t5pmetzk/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 200, in run_commands
          dist.run_commands()
        File "/tmp/pip-build-env-t5pmetzk/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/tmp/pip-build-env-t5pmetzk/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 976, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-t5pmetzk/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-t5pmetzk/overlay/lib/python3.9/site-packages/wheel/bdist_wheel.py", line 368, in run
          self.run_command("build")
        File "/tmp/pip-build-env-t5pmetzk/overlay/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
          self.distribution.run_command(command)
        File "/tmp/pip-build-env-t5pmetzk/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 976, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-t5pmetzk/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-t5pmetzk/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build.py", line 132, in run
          self.run_command(cmd_name)
        File "/tmp/pip-build-env-t5pmetzk/overlay/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
          self.distribution.run_command(command)
        File "/tmp/pip-build-env-t5pmetzk/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 976, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-t5pmetzk/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "<string>", line 67, in run
        File "/home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/subprocess.py", line 528, in run
          raise CalledProcessError(retcode, process.args,
      subprocess.CalledProcessError: Command '['cmake', '--build', '/media/dicarlod/SSD_2/diego/Code/NeuralSteerer/nsteerer/third_party/sphericart/build/cmake-build', '--parallel', '--target', 'install']' returned non-zero exit status 2.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for sphericart
  Building wheel for sphericart-jax (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for sphericart-jax (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [94 lines of output]
      /tmp/pip-build-env-yjucuzmg/overlay/lib/python3.9/site-packages/setuptools/dist.py:320: InformationOnly: Normalizing '0.4.0-beta1' to '0.4.0b1'
        self.metadata.version = self._normalize_version(self.metadata.version)
      running bdist_wheel
      running build
      running build_py
      copying python/sphericart/jax/utils.py -> build/lib.linux-x86_64-cpython-39/sphericart/jax
      copying python/sphericart/jax/ddsph.py -> build/lib.linux-x86_64-cpython-39/sphericart/jax
      copying python/sphericart/jax/dsph.py -> build/lib.linux-x86_64-cpython-39/sphericart/jax
      copying python/sphericart/jax/__init__.py -> build/lib.linux-x86_64-cpython-39/sphericart/jax
      copying python/sphericart/jax/spherical_harmonics.py -> build/lib.linux-x86_64-cpython-39/sphericart/jax
      copying python/sphericart/jax/sph.py -> build/lib.linux-x86_64-cpython-39/sphericart/jax
      running egg_info
      writing python/sphericart_jax.egg-info/PKG-INFO
      writing dependency_links to python/sphericart_jax.egg-info/dependency_links.txt
      writing requirements to python/sphericart_jax.egg-info/requires.txt
      writing top-level names to python/sphericart_jax.egg-info/top_level.txt
      reading manifest file 'python/sphericart_jax.egg-info/SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      writing manifest file 'python/sphericart_jax.egg-info/SOURCES.txt'
      running build_ext
      CMake Warning (dev) at /tmp/pip-build-env-yjucuzmg/overlay/lib/python3.9/site-packages/pybind11/share/cmake/pybind11/FindPythonLibsNew.cmake:101 (message):
        Policy CMP0148 is not set: The FindPythonInterp and FindPythonLibs modules
        are removed.  Run "cmake --help-policy CMP0148" for policy details.  Use
        the cmake_policy command to set the policy and suppress this warning, or
        preferably upgrade to using FindPython, either by calling it explicitly
        before pybind11, or by setting PYBIND11_FINDPYTHON ON before pybind11.
      Call Stack (most recent call first):
        /tmp/pip-build-env-yjucuzmg/overlay/lib/python3.9/site-packages/pybind11/share/cmake/pybind11/pybind11Tools.cmake:50 (find_package)
        /tmp/pip-build-env-yjucuzmg/overlay/lib/python3.9/site-packages/pybind11/share/cmake/pybind11/pybind11Common.cmake:202 (include)
        /tmp/pip-build-env-yjucuzmg/overlay/lib/python3.9/site-packages/pybind11/share/cmake/pybind11/pybind11Config.cmake:250 (include)
        CMakeLists.txt:34 (find_package)
      This warning is for project developers.  Use -Wno-dev to suppress it.

      -- Found pybind11: /tmp/pip-build-env-yjucuzmg/overlay/lib/python3.9/site-packages/pybind11/include (found version "2.13.0")
      -- OpenMP is enabled
      -- march=native is enabled
      -- Configuring done (0.2s)
      -- Generating done (0.0s)
      -- Build files have been written to: /media/dicarlod/SSD_2/diego/Code/NeuralSteerer/nsteerer/third_party/sphericart/sphericart-jax/build/cmake-build
      [ 20%] Building CUDA object sphericart/CMakeFiles/sphericart.dir/src/cuda_base.cu.o
      [ 20%] Building CUDA object sphericart/CMakeFiles/sphericart.dir/src/sphericart_cuda.cu.o
      nvcc fatal   : Unsupported gpu architecture 'compute_86'
      make[2]: *** [sphericart/CMakeFiles/sphericart.dir/build.make:119: sphericart/CMakeFiles/sphericart.dir/src/sphericart_cuda.cu.o] Error 1
      make[2]: *** Waiting for unfinished jobs....
      nvcc fatal   : Unsupported gpu architecture 'compute_86'
      make[2]: *** [sphericart/CMakeFiles/sphericart.dir/build.make:104: sphericart/CMakeFiles/sphericart.dir/src/cuda_base.cu.o] Error 1
      make[1]: *** [CMakeFiles/Makefile2:153: sphericart/CMakeFiles/sphericart.dir/all] Error 2
      make: *** [Makefile:136: all] Error 2
      Traceback (most recent call last):
        File "/home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
          return _build_backend().build_wheel(wheel_directory, config_settings,
        File "/tmp/pip-build-env-yjucuzmg/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 415, in build_wheel
          return self._build_with_temp_dir(
        File "/tmp/pip-build-env-yjucuzmg/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 397, in _build_with_temp_dir
          self.run_setup()
        File "/tmp/pip-build-env-yjucuzmg/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 313, in run_setup
          exec(code, locals())
        File "<string>", line 76, in <module>
        File "/tmp/pip-build-env-yjucuzmg/overlay/lib/python3.9/site-packages/setuptools/__init__.py", line 103, in setup
          return distutils.core.setup(**attrs)
        File "/tmp/pip-build-env-yjucuzmg/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 184, in setup
          return run_commands(dist)
        File "/tmp/pip-build-env-yjucuzmg/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 200, in run_commands
          dist.run_commands()
        File "/tmp/pip-build-env-yjucuzmg/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/tmp/pip-build-env-yjucuzmg/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 976, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-yjucuzmg/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-yjucuzmg/overlay/lib/python3.9/site-packages/setuptools/command/bdist_wheel.py", line 373, in run
          self.run_command("build")
        File "/tmp/pip-build-env-yjucuzmg/overlay/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
          self.distribution.run_command(command)
        File "/tmp/pip-build-env-yjucuzmg/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 976, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-yjucuzmg/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-yjucuzmg/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build.py", line 132, in run
          self.run_command(cmd_name)
        File "/tmp/pip-build-env-yjucuzmg/overlay/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
          self.distribution.run_command(command)
        File "/tmp/pip-build-env-yjucuzmg/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 976, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-yjucuzmg/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "<string>", line 55, in run
        File "/home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/subprocess.py", line 528, in run
          raise CalledProcessError(retcode, process.args,
      subprocess.CalledProcessError: Command '['cmake', '--build', '/media/dicarlod/SSD_2/diego/Code/NeuralSteerer/nsteerer/third_party/sphericart/sphericart-jax/build/cmake-build', '--parallel', '--target', 'install']' returned non-zero exit status 2.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for sphericart-jax
Failed to build sphericart sphericart-jax
ERROR: Could not build wheels for sphericart, sphericart-jax, which is required to install pyproject.toml-based projects

Here are the required outputs: nvidia-smi

Wed Jun 26 19:22:31 2024       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 555.42.02              Driver Version: 555.42.02      CUDA Version: 12.5     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA RTX A6000               Off |   00000000:02:00.0 Off |                  Off |
| 30%   44C    P8             21W /  300W |      24MiB /  49140MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
|   1  Quadro P620                    Off |   00000000:B3:00.0  On |                  N/A |
| 51%   64C    P0             N/A /  N/A  |    1823MiB /   2048MiB |     35%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A      2045      G   /usr/lib/xorg/Xorg                              4MiB |
|    0   N/A  N/A      2394      G   /usr/lib/xorg/Xorg                              4MiB |
|    1   N/A  N/A      2045      G   /usr/lib/xorg/Xorg                             91MiB |
|    1   N/A  N/A      2394      G   /usr/lib/xorg/Xorg                            333MiB |
|    1   N/A  N/A      2535      G   /usr/bin/gnome-shell                          200MiB |
|    1   N/A  N/A      3409      G   /usr/lib/firefox/firefox                      626MiB |
|    1   N/A  N/A      5821      G   ...yOnDemand --variations-seed-version         66MiB |
|    1   N/A  N/A      6262      G   ...erProcess --variations-seed-version        430MiB |
|    1   N/A  N/A    136636      G   ...erProcess --variations-seed-version         60MiB |
+-----------------------------------------------------------------------------------------+

nvcc --version

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Sun_Jul_28_19:07:16_PDT_2019
Cuda compilation tools, release 10.1, V10.1.243
nickjbrowning commented 5 months ago

It looks like your CUDA installation is not correct.

You're trying to compile with a nvcc binary which comes from a CUDA 10.1 installation, but nvidia-smi (correctly) displays 12.5.

You should make sure that both are consistent.

RTX 30 series cards (sm_86) came after the release of CUDA 10.1, which is why nvcc is complaining here.

Chutlhu commented 5 months ago

I see. I was hoping that using the conda environment, I wouldn't have to worry about this. I will dig into this and come back asap Thank you very much for the explanation and your time

nickjbrowning commented 5 months ago

You probably can use conda, but you still need to make sure that the relevant paths are being correctly set (CUDA_HOME, PATH and LD_LIBRARY_FLAGS), and also to make sure you don't have multiple versions of the cudatoolkit laying around inside these paths.

If you want to keep particular conda environments isolated, you can also update the activate.d/activate.sh and deactivate.d/deactivate.sh scripts in the path/to/conda/env/etc folder (I am probably wrong with this particular path, so I'd look for it).

I prefer to install the cudatoolkit manually, and to make sure this is my only source of CUDA.

Chutlhu commented 5 months ago

Dear all, I installed the correct nvcc:

$ which nvcc      
/home/dicarlod/anaconda3/envs/python39cuda12/bin/nvcc
$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Wed_Apr_17_19:19:55_PDT_2024
Cuda compilation tools, release 12.5, V12.5.40
Build cuda_12.5.r12.5/compiler.34177558_0

but it seems that the compilation still fails

$ pip install '.[jax]'
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Processing /media/dicarlod/SSD_2/diego/Code/NeuralSteerer/nsteerer/third_party/sphericart
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: numpy in /home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/site-packages (from sphericart==0.4.0b1) (1.26.4)
Processing ./sphericart-jax (from sphericart==0.4.0b1)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: jax>=0.4.18 in /home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/site-packages (from sphericart-jax@ file:///media/dicarlod/SSD_2/diego/Code/NeuralSteerer/nsteerer/third_party/sphericart/sphericart-jax?50f1cf12-ca73-4b9e-bd62-e761f31286c8->sphericart==0.4.0b1) (0.4.29)
Requirement already satisfied: ml-dtypes>=0.4.0 in /home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/site-packages (from jax>=0.4.18->sphericart-jax@ file:///media/dicarlod/SSD_2/diego/Code/NeuralSteerer/nsteerer/third_party/sphericart/sphericart-jax?50f1cf12-ca73-4b9e-bd62-e761f31286c8->sphericart==0.4.0b1) (0.4.0)
Requirement already satisfied: opt-einsum in /home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/site-packages (from jax>=0.4.18->sphericart-jax@ file:///media/dicarlod/SSD_2/diego/Code/NeuralSteerer/nsteerer/third_party/sphericart/sphericart-jax?50f1cf12-ca73-4b9e-bd62-e761f31286c8->sphericart==0.4.0b1) (3.3.0)
Requirement already satisfied: scipy>=1.9 in /home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/site-packages (from jax>=0.4.18->sphericart-jax@ file:///media/dicarlod/SSD_2/diego/Code/NeuralSteerer/nsteerer/third_party/sphericart/sphericart-jax?50f1cf12-ca73-4b9e-bd62-e761f31286c8->sphericart==0.4.0b1) (1.13.1)
Requirement already satisfied: importlib-metadata>=4.6 in /home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/site-packages (from jax>=0.4.18->sphericart-jax@ file:///media/dicarlod/SSD_2/diego/Code/NeuralSteerer/nsteerer/third_party/sphericart/sphericart-jax?50f1cf12-ca73-4b9e-bd62-e761f31286c8->sphericart==0.4.0b1) (7.1.0)
Requirement already satisfied: zipp>=0.5 in /home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/site-packages (from importlib-metadata>=4.6->jax>=0.4.18->sphericart-jax@ file:///media/dicarlod/SSD_2/diego/Code/NeuralSteerer/nsteerer/third_party/sphericart/sphericart-jax?50f1cf12-ca73-4b9e-bd62-e761f31286c8->sphericart==0.4.0b1) (3.19.2)
Building wheels for collected packages: sphericart, sphericart-jax
  Building wheel for sphericart (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for sphericart (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [82 lines of output]
      /tmp/pip-build-env-3tznknwn/overlay/lib/python3.9/site-packages/setuptools/dist.py:320: InformationOnly: Normalizing '0.4.0-beta1' to '0.4.0b1'
        self.metadata.version = self._normalize_version(self.metadata.version)
      running bdist_wheel
      running build
      running build_py
      copying python/src/sphericart/_c_lib.py -> build/lib.linux-x86_64-cpython-39/sphericart
      copying python/src/sphericart/__init__.py -> build/lib.linux-x86_64-cpython-39/sphericart
      copying python/src/sphericart/spherical_harmonics.py -> build/lib.linux-x86_64-cpython-39/sphericart
      running egg_info
      writing python/src/sphericart.egg-info/PKG-INFO
      writing dependency_links to python/src/sphericart.egg-info/dependency_links.txt
      writing requirements to python/src/sphericart.egg-info/requires.txt
      writing top-level names to python/src/sphericart.egg-info/top_level.txt
      reading manifest file 'python/src/sphericart.egg-info/SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      warning: no files found matching 'LICENSE'
      adding license file 'LICENSE-APACHE'
      adding license file 'LICENSE-MIT'
      adding license file 'LICENSE.md'
      writing manifest file 'python/src/sphericart.egg-info/SOURCES.txt'
      running build_ext
      -- OpenMP is enabled
      -- march=native is enabled
      -- Configuring done (0.0s)
      -- Generating done (0.0s)
      -- Build files have been written to: /media/dicarlod/SSD_2/diego/Code/NeuralSteerer/nsteerer/third_party/sphericart/build/cmake-build
      [ 20%] Building CUDA object CMakeFiles/sphericart.dir/src/cuda_base.cu.o
      [ 40%] Building CUDA object CMakeFiles/sphericart.dir/src/sphericart_cuda.cu.o
      nvcc fatal   : Unsupported gpu architecture 'compute_86'
      nvcc fatal   : Unsupported gpu architecture 'compute_86'
      make[2]: *** [CMakeFiles/sphericart.dir/build.make:104: CMakeFiles/sphericart.dir/src/cuda_base.cu.o] Error 1
      make[2]: *** Waiting for unfinished jobs....
      make[2]: *** [CMakeFiles/sphericart.dir/build.make:119: CMakeFiles/sphericart.dir/src/sphericart_cuda.cu.o] Error 1
      make[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/sphericart.dir/all] Error 2
      make: *** [Makefile:146: all] Error 2
      sphericart: CUDA_HOME:  None
      Traceback (most recent call last):
        File "/home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
          return _build_backend().build_wheel(wheel_directory, config_settings,
        File "/tmp/pip-build-env-3tznknwn/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 415, in build_wheel
          return self._build_with_temp_dir(
        File "/tmp/pip-build-env-3tznknwn/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 397, in _build_with_temp_dir
          self.run_setup()
        File "/tmp/pip-build-env-3tznknwn/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 313, in run_setup
          exec(code, locals())
        File "<string>", line 120, in <module>
        File "/tmp/pip-build-env-3tznknwn/overlay/lib/python3.9/site-packages/setuptools/__init__.py", line 103, in setup
          return distutils.core.setup(**attrs)
        File "/tmp/pip-build-env-3tznknwn/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 184, in setup
          return run_commands(dist)
        File "/tmp/pip-build-env-3tznknwn/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 200, in run_commands
          dist.run_commands()
        File "/tmp/pip-build-env-3tznknwn/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/tmp/pip-build-env-3tznknwn/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 976, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-3tznknwn/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-3tznknwn/overlay/lib/python3.9/site-packages/wheel/bdist_wheel.py", line 368, in run
          self.run_command("build")
        File "/tmp/pip-build-env-3tznknwn/overlay/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
          self.distribution.run_command(command)
        File "/tmp/pip-build-env-3tznknwn/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 976, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-3tznknwn/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-3tznknwn/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build.py", line 132, in run
          self.run_command(cmd_name)
        File "/tmp/pip-build-env-3tznknwn/overlay/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
          self.distribution.run_command(command)
        File "/tmp/pip-build-env-3tznknwn/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 976, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-3tznknwn/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "<string>", line 67, in run
        File "/home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/subprocess.py", line 528, in run
          raise CalledProcessError(retcode, process.args,
      subprocess.CalledProcessError: Command '['cmake', '--build', '/media/dicarlod/SSD_2/diego/Code/NeuralSteerer/nsteerer/third_party/sphericart/build/cmake-build', '--parallel', '--target', 'install']' returned non-zero exit status 2.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for sphericart
  Building wheel for sphericart-jax (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for sphericart-jax (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [94 lines of output]
      /tmp/pip-build-env-crdaopvb/overlay/lib/python3.9/site-packages/setuptools/dist.py:320: InformationOnly: Normalizing '0.4.0-beta1' to '0.4.0b1'
        self.metadata.version = self._normalize_version(self.metadata.version)
      running bdist_wheel
      running build
      running build_py
      copying python/sphericart/jax/utils.py -> build/lib.linux-x86_64-cpython-39/sphericart/jax
      copying python/sphericart/jax/ddsph.py -> build/lib.linux-x86_64-cpython-39/sphericart/jax
      copying python/sphericart/jax/dsph.py -> build/lib.linux-x86_64-cpython-39/sphericart/jax
      copying python/sphericart/jax/__init__.py -> build/lib.linux-x86_64-cpython-39/sphericart/jax
      copying python/sphericart/jax/spherical_harmonics.py -> build/lib.linux-x86_64-cpython-39/sphericart/jax
      copying python/sphericart/jax/sph.py -> build/lib.linux-x86_64-cpython-39/sphericart/jax
      running egg_info
      writing python/sphericart_jax.egg-info/PKG-INFO
      writing dependency_links to python/sphericart_jax.egg-info/dependency_links.txt
      writing requirements to python/sphericart_jax.egg-info/requires.txt
      writing top-level names to python/sphericart_jax.egg-info/top_level.txt
      reading manifest file 'python/sphericart_jax.egg-info/SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      writing manifest file 'python/sphericart_jax.egg-info/SOURCES.txt'
      running build_ext
      CMake Warning (dev) at /tmp/pip-build-env-crdaopvb/overlay/lib/python3.9/site-packages/pybind11/share/cmake/pybind11/FindPythonLibsNew.cmake:101 (message):
        Policy CMP0148 is not set: The FindPythonInterp and FindPythonLibs modules
        are removed.  Run "cmake --help-policy CMP0148" for policy details.  Use
        the cmake_policy command to set the policy and suppress this warning, or
        preferably upgrade to using FindPython, either by calling it explicitly
        before pybind11, or by setting PYBIND11_FINDPYTHON ON before pybind11.
      Call Stack (most recent call first):
        /tmp/pip-build-env-crdaopvb/overlay/lib/python3.9/site-packages/pybind11/share/cmake/pybind11/pybind11Tools.cmake:50 (find_package)
        /tmp/pip-build-env-crdaopvb/overlay/lib/python3.9/site-packages/pybind11/share/cmake/pybind11/pybind11Common.cmake:202 (include)
        /tmp/pip-build-env-crdaopvb/overlay/lib/python3.9/site-packages/pybind11/share/cmake/pybind11/pybind11Config.cmake:250 (include)
        CMakeLists.txt:34 (find_package)
      This warning is for project developers.  Use -Wno-dev to suppress it.

      -- Found pybind11: /tmp/pip-build-env-crdaopvb/overlay/lib/python3.9/site-packages/pybind11/include (found version "2.13.1")
      -- OpenMP is enabled
      -- march=native is enabled
      -- Configuring done (0.2s)
      -- Generating done (0.0s)
      -- Build files have been written to: /media/dicarlod/SSD_2/diego/Code/NeuralSteerer/nsteerer/third_party/sphericart/sphericart-jax/build/cmake-build
      [ 10%] Building CUDA object sphericart/CMakeFiles/sphericart.dir/src/cuda_base.cu.o
      [ 20%] Building CUDA object sphericart/CMakeFiles/sphericart.dir/src/sphericart_cuda.cu.o
      nvcc fatal   : Unsupported gpu architecture 'compute_86'
      make[2]: *** [sphericart/CMakeFiles/sphericart.dir/build.make:104: sphericart/CMakeFiles/sphericart.dir/src/cuda_base.cu.o] Error 1
      make[2]: *** Waiting for unfinished jobs....
      nvcc fatal   : Unsupported gpu architecture 'compute_86'
      make[2]: *** [sphericart/CMakeFiles/sphericart.dir/build.make:119: sphericart/CMakeFiles/sphericart.dir/src/sphericart_cuda.cu.o] Error 1
      make[1]: *** [CMakeFiles/Makefile2:153: sphericart/CMakeFiles/sphericart.dir/all] Error 2
      make: *** [Makefile:136: all] Error 2
      Traceback (most recent call last):
        File "/home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
          return _build_backend().build_wheel(wheel_directory, config_settings,
        File "/tmp/pip-build-env-crdaopvb/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 415, in build_wheel
          return self._build_with_temp_dir(
        File "/tmp/pip-build-env-crdaopvb/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 397, in _build_with_temp_dir
          self.run_setup()
        File "/tmp/pip-build-env-crdaopvb/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 313, in run_setup
          exec(code, locals())
        File "<string>", line 76, in <module>
        File "/tmp/pip-build-env-crdaopvb/overlay/lib/python3.9/site-packages/setuptools/__init__.py", line 103, in setup
          return distutils.core.setup(**attrs)
        File "/tmp/pip-build-env-crdaopvb/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 184, in setup
          return run_commands(dist)
        File "/tmp/pip-build-env-crdaopvb/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 200, in run_commands
          dist.run_commands()
        File "/tmp/pip-build-env-crdaopvb/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/tmp/pip-build-env-crdaopvb/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 976, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-crdaopvb/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-crdaopvb/overlay/lib/python3.9/site-packages/setuptools/command/bdist_wheel.py", line 373, in run
          self.run_command("build")
        File "/tmp/pip-build-env-crdaopvb/overlay/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
          self.distribution.run_command(command)
        File "/tmp/pip-build-env-crdaopvb/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 976, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-crdaopvb/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-crdaopvb/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build.py", line 132, in run
          self.run_command(cmd_name)
        File "/tmp/pip-build-env-crdaopvb/overlay/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
          self.distribution.run_command(command)
        File "/tmp/pip-build-env-crdaopvb/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 976, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-crdaopvb/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "<string>", line 55, in run
        File "/home/dicarlod/anaconda3/envs/python39cuda12/lib/python3.9/subprocess.py", line 528, in run
          raise CalledProcessError(retcode, process.args,
      subprocess.CalledProcessError: Command '['cmake', '--build', '/media/dicarlod/SSD_2/diego/Code/NeuralSteerer/nsteerer/third_party/sphericart/sphericart-jax/build/cmake-build', '--parallel', '--target', 'install']' returned non-zero exit status 2.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for sphericart-jax
Failed to build sphericart sphericart-jax
ERROR: Could not build wheels for sphericart, sphericart-jax, which is required to install pyproject.toml-based projects

it seems that it is not using the correct nvcc, is it? I tried with both all and native as suggested before

Chutlhu commented 5 months ago

Update: Now it worked with pip install --pre sphericart[jax] the problem was indeed with the nvcc, so having the right nvcc and the "--pre" solved the issues. However, the above error in the compilation from source failed

Luthaf commented 5 months ago

However, the above error in the compilation from source failed

If you change your compilers, nvcc included, you need to remove the local build cache (in the build/ folder). Installing with pip creates a separate temporary build cache, hence why you managed to compile it in this case!