RuntimeError: Cannot initialize CUDA without ATen_cuda library

[jazberna1]

Hello!

I am getting the following error the following line of the tribolium_cartograph.ipynb notebook

flow = timelapse_flow(image, store_or_path="flow.zarr", n_scales=2, lr=1e-2, num_iterations=2_000)

This is the actual error:

RuntimeError: Cannot initialize CUDA without ATen_cuda library. PyTorch splits its backend into two shared libraries: a CPU library and a CUDA library; this error has occurred because you are trying to use some CUDA functionality, but the CUDA library has not been loaded by the dynamic linker for some reason.  The CUDA library MUST be loaded, EVEN IF you don't directly use any symbols from the CUDA library! One common culprit is a lack of -INCLUDE:?warp_size@cuda@at@@YAHXZ in your link arguments; many dynamic linkers will delete dynamic library dependencies if you don't depend on any of their symbols.  You can check if this has occurred by using link on your binary to see if there is a dependency on *_cuda.dll library.

I was able to follow the ultrack installation instructions but somehow I am still missing something.

I was wondering if you could please give me hint to fix it, any help will be much appreciated.

Thanks Jorge

[JoOkuma]

Hi @jazberna1,

How did you install ultrack / pytorch and which platform are you using? Windows?

Cheers, Jordao

[jazberna1]

Hi!

Yes, I am on Windows 10.

I installed ultrack following the isntructions here: https://royerlab.github.io/ultrack/

create the enviroment conda create -n ultrack python=3.11 higra gurobi pytorch pyqt -c pytorch -c gurobi -c conda-forge

activate the environment and install ultrack conda activate ultrack pip install ultrack

GPU acceleration conda install pytorch-cuda -c pytorch -c nvidia conda install cupy -c conda-forge

The only thing I had to change was the cucim installation as the comand given was not working: pip install "git+https://github.com/rapidsai/cucim.git#egg=cucim&subdirectory=python/cucim" I changed that command for this other one that I got after asking in the cucim repo, and which worked for me: pip install -e "git+https://github.com/rapidsai/cucim.git@v22.04.00#egg=cucim&subdirectory=python/cucim" scipy scikit-image

This is the error I got running the cucim install command in the ultrack page, the one I said above that didn't work for me, just in case is of any help:

Collecting cicum
  Cloning https://github.com/rapidsai/cucim.git to c:\users\malieva\appdata\local\temp\pip-install-rli45bl_\cucim_dfc45313f4184775842ca13fb16342ae
  Running command git clone --filter=blob:none --quiet https://github.com/rapidsai/cucim.git 'C:\Users\malieva\AppData\Local\Temp\pip-install-rli45bl_\cucim_dfc45313f4184775842ca13fb16342ae'
  Resolved https://github.com/rapidsai/cucim.git to commit 7593ea739efefbda4a286db52e0c13b299f5fe18
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [29 lines of output]
      Traceback (most recent call last):
        File "C:\Users\malieva\miniconda3\envs\ultrack\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
          main()
        File "C:\Users\malieva\miniconda3\envs\ultrack\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\malieva\miniconda3\envs\ultrack\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\malieva\AppData\Local\Temp\pip-build-env-qf98k6tf\overlay\Lib\site-packages\rapids_build_backend\impls.py", line 257, in get_requires_for_build_wheel
          with _edit_pyproject(config), _write_git_commits(config, project_name):
        File "C:\Users\malieva\miniconda3\envs\ultrack\Lib\contextlib.py", line 137, in __enter__
          return next(self.gen)
                 ^^^^^^^^^^^^^^
        File "C:\Users\malieva\AppData\Local\Temp\pip-build-env-qf98k6tf\overlay\Lib\site-packages\rapids_build_backend\impls.py", line 155, in _edit_pyproject
          cuda_version_major, cuda_version_minor = _get_cuda_version()
                                                   ^^^^^^^^^^^^^^^^^^^
        File "C:\Users\malieva\AppData\Local\Temp\pip-build-env-qf98k6tf\overlay\Lib\site-packages\rapids_build_backend\impls.py", line 63, in _get_cuda_version
          nvcc_exists = subprocess.run(["which", "nvcc"], capture_output=True).returncode == 0
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\malieva\miniconda3\envs\ultrack\Lib\subprocess.py", line 548, in run
          with Popen(*popenargs, **kwargs) as process:
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\malieva\miniconda3\envs\ultrack\Lib\subprocess.py", line 1026, in __init__
          self._execute_child(args, executable, preexec_fn, close_fds,
        File "C:\Users\malieva\miniconda3\envs\ultrack\Lib\subprocess.py", line 1538, in _execute_child
          hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      FileNotFoundError: [WinError 2] El sistema no puede encontrar el archivo especificado
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Many thanks for your help!! Jorge

[jazberna1]

Hi,

I did some reserach and according to this page (https://pytorch.org/get-started/locally/) the command to install torch on windows shoud be:

pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118

By installing torch that way I'be able to run the tribolium_cartograph.ipynb notebook up to this point

flow = timelapse_flow(image, store_or_path="flow.zarr", n_scales=2, lr=1e-2, num_iterations=2_000)
viewer.add_image(
    flow,
    contrast_limits=(-0.001, 0.001),
    colormap="turbo",
    visible=False,
    scale=(4,) * 3,
    channel_axis=1,
    name="flow field",
)

Then it fails of the next line which I reported in another ticket https://github.com/royerlab/ultrack/issues/157 trajectory = advenct_from_quasi_random(flow, detection.shape[-3:], n_samples=1000)

Thanks Jorge

[JoOkuma]

HI @jazberna1 , I'm glad you found the solution. Can we close this issue and follow through on #157?

Thanks

[jazberna1]

Hi @JoOkuma Sure I close it

Thanks!! Jorge