search

lasopablo / freesurfer-freesurfer-dev-mri_WMHsynthseg

Project at Harvard (HMS and MGH) for Deep Learning-powered WMH quantification. Please refer to the official website for more recent information.
https://surfer.nmr.mgh.harvard.edu/fswiki/WMH-SynthSeg
2 stars 2 forks source link

Exception: Input does not exist #2

Closed zhaoliang0302 closed 6 months ago

zhaoliang0302 commented 6 months ago

Hi, I am currently facing an issue while running the mri_WMHsynthseg function. The specific error message I am encountering is:Exception: Input does not exist.

I have verified that the input directory specified with the --i parameter does exist. However, I am unable to resolve this issue. I am reaching out to seek your assistance in understanding and rectifying this problem.

$ mri_WMHsynthseg --i /home/dbs/Data/MRI_converted/I22/I223.nii --o /home/dbs/Data/output/ --threads 24
Traceback (most recent call last):
  File "/home/dbs/Software/freesurfer/python/packages/WMHSynthSeg/inference.py", line 284, in <module>
    main()
  File "/home/dbs/Software/freesurfer/python/packages/WMHSynthSeg/inference.py", line 37, in main
    raise Exception('Input does not exist')
Exception: Input does not exist

Thanks.

lasopablo commented 6 months ago

Dear Liang Zhao,

Thank you for reaching out and for your detailed description of the issue you're encountering with the WMH-SynthSeg function in FreeSurfer.

Given the error message "Input does not exist," and considering that you have already verified the existence of the input directory and file, a few additional checks could help resolve the issue:

Check File Permissions: Please ensure that the user running the script has the necessary permissions to access /home/dbs/Data/MRI_converted/I22/I223.nii. You can verify this by running ls -l /home/dbs/Data/MRI_converted/I22/I223.nii to check the permissions. Absolute vs. Relative Paths: Confirm that the script is correctly resolving the path to the input file. If possible, try providing an absolute path to the script, which you've done, but also ensure there are no symbolic links or path-related issues. Environment Configuration: Verify that your environment is correctly set up, especially the FREESURFER_HOME variable. Ensure it is accessible in the context the script is run, which you can check by echoing the variable (echo $FREESURFER_HOME) before running the script. If these steps do not resolve the issue, it might be helpful to include additional logging within the script to confirm the script's perceived path to the input file right before the existence check. This could provide further insights into any discrepancies between the expected and actual paths.

Please let me know if any of these suggestions help or if the problem persists. Further information, such as the operating system details and the version of FreeSurfer being used, could also be beneficial for deeper troubleshooting.

Best regards,

zhaoliang0302 commented 6 months ago

Hi,

Following the steps you provided, I managed to resolve the previous error "Input does not exist." However, I encountered a new error related to the CUDA environment configuration (I guess). It seems that the issue persists despite my attempts to configure the analysis environment on my own. As someone without experience in this area, I find it challenging to troubleshoot further.

$ mri_WMHsynthseg --i /home/dbs/Data/MRI_converted/I223766/ --o /home/dbs/Data/output/ --threads 24
Arguments seem correct; loading Python packages...
Traceback (most recent call last):
  File "/home/dbs/Software/freesurfer/python/lib/python3.8/site-packages/torch/__init__.py", line 174, in _load_global_deps
    ctypes.CDLL(lib_path, mode=ctypes.RTLD_GLOBAL)
  File "/home/dbs/Software/freesurfer/python/lib/python3.8/ctypes/__init__.py", line 373, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: libcufft.so.11: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/dbs/Software/freesurfer/python/packages/WMHSynthSeg/inference.py", line 284, in <module>
    main()
  File "/home/dbs/Software/freesurfer/python/packages/WMHSynthSeg/inference.py", line 76, in main
    import torch
  File "/home/dbs/Software/freesurfer/python/lib/python3.8/site-packages/torch/__init__.py", line 234, in <module>
    _load_global_deps()
  File "/home/dbs/Software/freesurfer/python/lib/python3.8/site-packages/torch/__init__.py", line 195, in _load_global_deps
    _preload_cuda_deps(lib_folder, lib_name)
  File "/home/dbs/Software/freesurfer/python/lib/python3.8/site-packages/torch/__init__.py", line 160, in _preload_cuda_deps
    raise ValueError(f"{lib_name} not found in the system path {sys.path}")
ValueError: libcublas.so.*[0-9] not found in the system path ['/home/dbs/Software/freesurfer/python/packages/WMHSynthSeg', '/home/dbs/Software/freesurfer/python/packages', '/home/dbs/Software/freesurfer/python/lib/python38.zip', '/home/dbs/Software/freesurfer/python/lib/python3.8', '/home/dbs/Software/freesurfer/python/lib/python3.8/lib-dynload', '/home/dbs/Software/freesurfer/python/lib/python3.8/site-packages']
$ cd /home/dbs/Software/
$ wget https://developer.download.nvidia.com/compute/cuda/12.2.2/local_installers/cuda_12.2.2_535.104.05_linux.run
$ sudo sh cuda_12.2.2_535.104.05_linux.run
$ nvcc -V

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Tue_Aug_15_22:02:13_PDT_2023
Cuda compilation tools, release 12.2, V12.2.140
Build cuda_12.2.r12.2/compiler.33191640_0
$ mri_WMHsynthseg --i /home/dbs/Data/MRI_converted/I223766/ --o /home/dbs/Data/output/ --threads 24
Arguments seem correct; loading Python packages...
Traceback (most recent call last):
  File "/home/dbs/Software/freesurfer/python/packages/WMHSynthSeg/inference.py", line 284, in <module>
    main()
  File "/home/dbs/Software/freesurfer/python/packages/WMHSynthSeg/inference.py", line 76, in main
    import torch
  File "/home/dbs/Software/freesurfer/python/lib/python3.8/site-packages/torch/__init__.py", line 235, in <module>
    from torch._C import *  # noqa: F403
ImportError: libcudnn.so.8: cannot open shared object file: No such file or directory

Could you please assist me in debugging this issue? Platform: Ubuntu 20.04 x86_64 in WMware. FreeSurfer is the latest development version.

lasopablo commented 6 months ago

The errors you're encountering are related to missing CUDA library files necessary for running GPU-accelerated applications, specifically those related to NVIDIA's CUDA toolkit and cuDNN (CUDA Deep Neural Network library). Do you have CUDA installed?

What happens, respectively, if you run the following?

export PATH=/usr/local/cuda-12.2/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-12.2/lib64:$LD_LIBRARY_PATH

and nvcc -V

zhaoliang0302 commented 6 months ago

Yes, I used the command you provided in the bashrc, and it seems that CUDA is installed and configured correctly. Also, after configuring cuDNN, the error "libcudnn.so.8: cannot open shared object file: No such file or directory" is no longer occurring. However, a new error has appeared: "libcupti.so.12: cannot open shared object file". Environment configuration is proving to be quite difficult and time-consuming for me. Could you please provide a Docker version of the FreeSurfer dev? I'm eager to incorporate mri_WMHsynthseg into my paper.

Thanks

lasopablo commented 6 months ago

Dear Liang Zhao,

Try running the following, and let me know if the same issue persists:

conda create --name pytorch python=3.7
conda activate pytorch
conda install pytorch==1.7.0 torchvision torchaudio cudatoolkit=11.0 -c pytorch*
lasopablo commented 6 months ago

Make sure to always work within your environment !