ClementPinard / Pytorch-Correlation-extension

Custom implementation of Corrleation Module
MIT License
414 stars 77 forks source link

Facing issue installing spatial correlation sampler #114

Open maniron214 opened 1 month ago

maniron214 commented 1 month ago

I am trying to install spatial correlation sampler packager for one of my application. I am installing it through pip install spatial-correlation-sampler

When ever I try to install I am getting an error

Traceback (most recent call last): File "C:\Program Files\Python311\Lib\site-packages\pip_vendor\pep517\in_process_in_process.py", line 351, in main() File "C:\Program Files\Python311\Lib\site-packages\pip_vendor\pep517\in_process_in_process.py", line 333, in main json_out['return_val'] = hook(**hook_input['kwargs']) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Program Files\Python311\Lib\site-packages\pip_vendor\pep517\in_process_in_process.py", line 152, in prepare_metadata_for_build_wheel return hook(metadata_directory, config_settings) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Program Files\Python311\Lib\site-packages\setuptools\build_meta.py", line 377, in prepare_metadata_for_build_wheel self.run_setup() File "C:\Program Files\Python311\Lib\site-packages\setuptools\build_meta.py", line 484, in run_setup self).run_setup(setup_script=setup_script) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Program Files\Python311\Lib\site-packages\setuptools\build_meta.py", line 335, in run_setup exec(code, locals()) File "", line 69, in File "", line 48, in launch_setup File "C:\Program Files\Python311\Lib\site-packages\torch\utils\cpp_extension.py", line 1076, in CUDAExtension library_dirs += library_paths(cuda=True) ^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Program Files\Python311\Lib\site-packages\torch\utils\cpp_extension.py", line 1214, in library_paths paths.append(_join_cuda_home(lib_dir)) ^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Program Files\Python311\Lib\site-packages\torch\utils\cpp_extension.py", line 2416, in _join_cuda_home raise OSError('CUDA_HOME environment variable is not set. ' OSError: CUDA_HOME environment variable is not set. Please set it to your CUDA install root.

My echo output

echo %CUDA_HOME% C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.6

Even though cuda tool kit is installed and this CUDA_HOME env is set I am getting the same error. I am trying this in windows os. If there is any solution kindly let me know

maniron214 commented 1 month ago

try this [REDACTED]

you may need to install the c compiler

Hi kindly let me know what is the URL about, As its getting blocked. I am actually installing VC++

ClementPinard commented 1 month ago

Please don't click blindly on bit.ly urls. I am almost certain this is some malware @nnimoyoefoki wants you to install, don't do it. I have removed the links. Besides, talking about a C compiler is completely off topic.

As for your problem, I cannot help you as much as I want, because I am not on windows. Have you tried pip install spatial-correlation-sampler --no-build-isolation this will try to match the cuda version with one from your own pytorch installation, and not the generic one from pypi

maniron214 commented 1 month ago

Please don't click blindly on bit.ly urls. I am almost certain this is some malware @nnimoyoefoki wants you to install, don't do it. I have removed the links. Besides, talking about a C compiler is completely off topic.

As for your problem, I cannot help you as much as I want, because I am not on windows. Have you tried pip install spatial-correlation-sampler --no-build-isolation this will try to match the cuda version with one from your own pytorch installation, and not the generic one from pypi

Sorry thought the comment was genuine, Thanks for responding, I tried this way also but the error remained same, before install the package what all to be verified, Cause I didnt face any issue while installing this in my ubuntu, but windows is what causing trouble

I tried installing c++ build tools from Microsoft but still no luck

Have someone installed spatial correlation sampler in windows ?

maniron214 commented 1 month ago

pip install spatial-correlation-sampler --verbose Using pip 24.2 from C:\Users\TestProfile\AppData\Roaming\Python\Python311\site-packages\pip (python 3.11) Defaulting to user installation because normal site-packages is not writeable Collecting spatial-correlation-sampler Using cached spatial_correlation_sampler-0.5.0.tar.gz (9.8 kB) Running command pip subprocess to install build dependencies Using pip 24.2 from C:\Users\TestProfile\AppData\Roaming\Python\Python311\site-packages\pip (python 3.11) Collecting setuptools Obtaining dependency information for setuptools from https://files.pythonhosted.org/packages/ff/ae/f19306b5a221f6a436d8f2238d5b80925004093fa3edea59835b514d9057/setuptools-75.1.0-py3-none-any.whl.metadata Using cached setuptools-75.1.0-py3-none-any.whl.metadata (6.9 kB) Collecting torch Obtaining dependency information for torch from https://files.pythonhosted.org/packages/5a/6a/775b93d6888c31f1f1fc457e4f5cc89f0984412d5dcdef792b8f2aa6e812/torch-2.4.1-cp311-cp311-win_amd64.whl.metadata Using cached torch-2.4.1-cp311-cp311-win_amd64.whl.metadata (27 kB) Collecting filelock (from torch) Obtaining dependency information for filelock from https://files.pythonhosted.org/packages/b9/f8/feced7779d755758a52d1f6635d990b8d98dc0a29fa568bbe0625f18fdf3/filelock-3.16.1-py3-none-any.whl.metadata Using cached filelock-3.16.1-py3-none-any.whl.metadata (2.9 kB) Collecting typing-extensions>=4.8.0 (from torch) Obtaining dependency information for typing-extensions>=4.8.0 from https://files.pythonhosted.org/packages/26/9f/ad63fc0248c5379346306f8668cda6e2e2e9c95e01216d2b8ffd9ff037d0/typing_extensions-4.12.2-py3-none-any.whl.metadata Using cached typing_extensions-4.12.2-py3-none-any.whl.metadata (3.0 kB) Collecting sympy (from torch) Obtaining dependency information for sympy from https://files.pythonhosted.org/packages/99/ff/c87e0622b1dadea79d2fb0b25ade9ed98954c9033722eb707053d310d4f3/sympy-1.13.3-py3-none-any.whl.metadata Using cached sympy-1.13.3-py3-none-any.whl.metadata (12 kB) Collecting networkx (from torch) Obtaining dependency information for networkx from https://files.pythonhosted.org/packages/38/e9/5f72929373e1a0e8d142a130f3f97e6ff920070f87f91c4e13e40e0fba5a/networkx-3.3-py3-none-any.whl.metadata Using cached networkx-3.3-py3-none-any.whl.metadata (5.1 kB) Collecting jinja2 (from torch) Obtaining dependency information for jinja2 from https://files.pythonhosted.org/packages/31/80/3a54838c3fb461f6fec263ebf3a3a41771bd05190238de3486aae8540c36/jinja2-3.1.4-py3-none-any.whl.metadata Using cached jinja2-3.1.4-py3-none-any.whl.metadata (2.6 kB) Collecting fsspec (from torch) Obtaining dependency information for fsspec from https://files.pythonhosted.org/packages/1d/a0/6aaea0c2fbea2f89bfd5db25fb1e3481896a423002ebe4e55288907a97a3/fsspec-2024.9.0-py3-none-any.whl.metadata Using cached fsspec-2024.9.0-py3-none-any.whl.metadata (11 kB) Collecting MarkupSafe>=2.0 (from jinja2->torch) Obtaining dependency information for MarkupSafe>=2.0 from https://files.pythonhosted.org/packages/b7/a2/c78a06a9ec6d04b3445a949615c4c7ed86a0b2eb68e44e7541b9d57067cc/MarkupSafe-2.1.5-cp311-cp311-win_amd64.whl.metadata Using cached MarkupSafe-2.1.5-cp311-cp311-win_amd64.whl.metadata (3.1 kB) Collecting mpmath<1.4,>=1.1.0 (from sympy->torch) Obtaining dependency information for mpmath<1.4,>=1.1.0 from https://files.pythonhosted.org/packages/43/e3/7d92a15f894aa0c9c4b49b8ee9ac9850d6e63b03c9c32c0367a13ae62209/mpmath-1.3.0-py3-none-any.whl.metadata Using cached mpmath-1.3.0-py3-none-any.whl.metadata (8.6 kB) Using cached setuptools-75.1.0-py3-none-any.whl (1.2 MB) Using cached torch-2.4.1-cp311-cp311-win_amd64.whl (199.4 MB) Using cached typing_extensions-4.12.2-py3-none-any.whl (37 kB) Using cached filelock-3.16.1-py3-none-any.whl (16 kB) Using cached fsspec-2024.9.0-py3-none-any.whl (179 kB) Using cached jinja2-3.1.4-py3-none-any.whl (133 kB) Using cached networkx-3.3-py3-none-any.whl (1.7 MB) Using cached sympy-1.13.3-py3-none-any.whl (6.2 MB) Using cached MarkupSafe-2.1.5-cp311-cp311-win_amd64.whl (17 kB) Using cached mpmath-1.3.0-py3-none-any.whl (536 kB) Installing collected packages: mpmath, typing-extensions, sympy, setuptools, networkx, MarkupSafe, fsspec, filelock, jinja2, torch Creating C:\Users\TestProfile\AppData\Local\Temp\pip-build-env-qzb3vqiz\overlay\Scripts Successfully installed MarkupSafe-2.1.5 filelock-3.16.1 fsspec-2024.9.0 jinja2-3.1.4 mpmath-1.3.0 networkx-3.3 setuptools-75.1.0 sympy-1.13.3 torch-2.4.1 typing-extensions-4.12.2 Installing build dependencies ... done Running command Getting requirements to build wheel C:\Users\TestProfile\AppData\Local\Temp\pip-build-env-qzb3vqiz\overlay\Lib\site-packages\torch_subclasses\functional_tensor.py:258: UserWarning: Failed to initialize NumPy: No module named 'numpy' (Triggered internally at C:\actions-runner_work\pytorch\pytorch\builder\windows\pytorch\torch\csrc\utils\tensor_numpy.cpp:84.) cpu = _conversion_method_template(device=torch.device("cpu")) Traceback (most recent call last): File "C:\Users\TestProfile\AppData\Roaming\Python\Python311\site-packages\pip_vendor\pyproject_hooks_in_process_in_process.py", line 353, in main() File "C:\Users\TestProfile\AppData\Roaming\Python\Python311\site-packages\pip_vendor\pyproject_hooks_in_process_in_process.py", line 335, in main json_out['return_val'] = hook(**hook_input['kwargs']) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\TestProfile\AppData\Roaming\Python\Python311\site-packages\pip_vendor\pyproject_hooks_in_process_in_process.py", line 118, in get_requires_for_build_wheel return hook(config_settings) ^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\TestProfile\AppData\Local\Temp\pip-build-env-qzb3vqiz\overlay\Lib\site-packages\setuptools\build_meta.py", line 332, in get_requires_for_build_wheel return self._get_build_requires(config_settings, requirements=[]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\TestProfile\AppData\Local\Temp\pip-build-env-qzb3vqiz\overlay\Lib\site-packages\setuptools\build_meta.py", line 302, in _get_build_requires self.run_setup() File "C:\Users\TestProfile\AppData\Local\Temp\pip-build-env-qzb3vqiz\overlay\Lib\site-packages\setuptools\build_meta.py", line 503, in run_setup super().run_setup(setup_script=setup_script) File "C:\Users\TestProfile\AppData\Local\Temp\pip-build-env-qzb3vqiz\overlay\Lib\site-packages\setuptools\build_meta.py", line 318, in run_setup exec(code, locals()) File "", line 69, in File "", line 48, in launch_setup File "C:\Users\TestProfile\AppData\Local\Temp\pip-build-env-qzb3vqiz\overlay\Lib\site-packages\torch\utils\cpp_extension.py", line 1076, in CUDAExtension library_dirs += library_paths(cuda=True) ^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\TestProfile\AppData\Local\Temp\pip-build-env-qzb3vqiz\overlay\Lib\site-packages\torch\utils\cpp_extension.py", line 1214, in library_paths paths.append(_join_cuda_home(lib_dir)) ^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\TestProfile\AppData\Local\Temp\pip-build-env-qzb3vqiz\overlay\Lib\site-packages\torch\utils\cpp_extension.py", line 2416, in _join_cuda_home raise OSError('CUDA_HOME environment variable is not set. ' OSError: CUDA_HOME environment variable is not set. Please set it to your CUDA install root. error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip. full command: 'C:\Program Files\Python311\python.exe' 'C:\Users\TestProfile\AppData\Roaming\Python\Python311\site-packages\pip_vendor\pyproject_hooks_in_process_in_process.py' get_requires_for_build_wheel 'C:\Users\testProfile\AppData\Local\Temp\tmpxwwfbjq9' cwd: C:\Users\TestProfile\AppData\Local\Temp\pip-install-a648lb52\spatial-correlation-sampler_46121e346ab549f3bd693c20ebacfa53 Getting requirements to build wheel ... error error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

HariWu1995 commented 1 month ago

Please don't click blindly on bit.ly urls. I am almost certain this is some malware @nnimoyoefoki wants you to install, don't do it. I have removed the links. Besides, talking about a C compiler is completely off topic.

As for your problem, I cannot help you as much as I want, because I am not on windows. Have you tried pip install spatial-correlation-sampler --no-build-isolation this will try to match the cuda version with one from your own pytorch installation, and not the generic one from pypi

Thanks a lot! It works for me!

maniron214 commented 1 month ago

Please don't click blindly on bit.ly urls. I am almost certain this is some malware @nnimoyoefoki wants you to install, don't do it. I have removed the links. Besides, talking about a C compiler is completely off topic. As for your problem, I cannot help you as much as I want, because I am not on windows. Have you tried pip install spatial-correlation-sampler --no-build-isolation this will try to match the cuda version with one from your own pytorch installation, and not the generic one from pypi

Thanks a lot! It works for me!

@HariWu1995 did it work for you in windows ?

HariWu1995 commented 1 month ago

Please don't click blindly on bit.ly urls. I am almost certain this is some malware @nnimoyoefoki wants you to install, don't do it. I have removed the links. Besides, talking about a C compiler is completely off topic. As for your problem, I cannot help you as much as I want, because I am not on windows. Have you tried pip install spatial-correlation-sampler --no-build-isolation this will try to match the cuda version with one from your own pytorch installation, and not the generic one from pypi

Thanks a lot! It works for me!

@HariWu1995 did it work for you in windows ?

Yes, it is.

My system info:

ClementPinard commented 1 month ago

Please don't click blindly on bit.ly urls. I am almost certain this is some malware @nnimoyoefoki wants you to install, don't do it. I have removed the links. Besides, talking about a C compiler is completely off topic. As for your problem, I cannot help you as much as I want, because I am not on windows. Have you tried pip install spatial-correlation-sampler --no-build-isolation this will try to match the cuda version with one from your own pytorch installation, and not the generic one from pypi

Thanks a lot! It works for me!

@HariWu1995 did it work for you in windows ?

The only thing I can recommand is you test the pytorch official extension repo : https://github.com/pytorch/extension-cpp

If you are able to build and install this one, this means the problem is in the code of the correlation sampler, if not, you might want to ask the question there.

At least you know what the error is exactly about : pytorch does not find the cuda version, although it should be able to find it from environment variable.

maniron214 commented 1 month ago

Please don't click blindly on bit.ly urls. I am almost certain this is some malware @nnimoyoefoki wants you to install, don't do it. I have removed the links. Besides, talking about a C compiler is completely off topic. As for your problem, I cannot help you as much as I want, because I am not on windows. Have you tried pip install spatial-correlation-sampler --no-build-isolation this will try to match the cuda version with one from your own pytorch installation, and not the generic one from pypi

Thanks a lot! It works for me!

@HariWu1995 did it work for you in windows ?

The only thing I can recommand is you test the pytorch official extension repo : https://github.com/pytorch/extension-cpp

If you are able to build and install this one, this means the problem is in the code of the correlation sampler, if not, you might want to ask the question there.

At least you know what the error is exactly about : pytorch does not find the cuda version, although it should be able to find it from environment variable.

I was able to download and install this

maniron214 commented 1 month ago

Please don't click blindly on bit.ly urls. I am almost certain this is some malware @nnimoyoefoki wants you to install, don't do it. I have removed the links. Besides, talking about a C compiler is completely off topic. As for your problem, I cannot help you as much as I want, because I am not on windows. Have you tried pip install spatial-correlation-sampler --no-build-isolation this will try to match the cuda version with one from your own pytorch installation, and not the generic one from pypi

Thanks a lot! It works for me!

@HariWu1995 did it work for you in windows ?

Yes, it is.

My system info:

  • System: Windows-10-10.0.22631-SP0
  • CUDA Version: 12.1
  • Python Version: 3.10.6
  • PyTorch Version: 2.4.0+cu121

I have python : 3.11 Torch: 2.4.1+cu124

Maybe I will try installing "PyTorch Version: 2.4.0+cu121" this version and try again

@HariWu1995 did you set any env variable ?

maniron214 commented 1 month ago

Please don't click blindly on bit.ly urls. I am almost certain this is some malware @nnimoyoefoki wants you to install, don't do it. I have removed the links. Besides, talking about a C compiler is completely off topic. As for your problem, I cannot help you as much as I want, because I am not on windows. Have you tried pip install spatial-correlation-sampler --no-build-isolation this will try to match the cuda version with one from your own pytorch installation, and not the generic one from pypi

Thanks a lot! It works for me!

@HariWu1995 did it work for you in windows ?

The only thing I can recommand is you test the pytorch official extension repo : https://github.com/pytorch/extension-cpp

If you are able to build and install this one, this means the problem is in the code of the correlation sampler, if not, you might want to ask the question there.

At least you know what the error is exactly about : pytorch does not find the cuda version, although it should be able to find it from environment variable.

spatial_correlation_sampler got installed when I install PyTorch Version: 2.4.0+cu121

@HariWu1995 and @ClementPinard thanks for the support

Now a new issue is arising

ModuleNotFoundError: No module named 'spatial_correlation_sampler_backend'

How this can be solved

maniron214 commented 1 month ago

Hi I tried building from the source but I am facing this issue

ModuleNotFoundError: No module named 'spatial_correlation_sampler_backend'

any idea on this

ImportError: DLL load failed while importing spatial_correlation_sampler_backend: The specified module could not be found.