Specific CUDA version needed to use pipeline?

[mathieuboudreau]

Hi there!

I just tried to run the pipeline on a dataset, and encountered an error,

F0909 18:32:25.907539 1837 im2col.cu:497] Check failed: error == cudaSuccess (209 vs. 0) no kernel image is available for execution on the device

That seems to indicate it might be related to a mismatch in CUDA version (expected from the Docker image vs actual): https://github.com/pytorch/pytorch/issues/31285

Are you aware if the Docker pipeline you build is only compatible with certain CUDA versions, and if so which one(s)?

Here's the full log of my pipeline run:

``` ubuntu@ip-172-31-3-79:~$ ls data HF00072_T2_6x_acquisitions.nii.gz HF00093_T2_3x_acquisitions.nii.gz License ubuntu@ip-172-31-3-79:~$ docker run --gpus=all --rm -it -v data:/InfantData --user $(id -u):$(id -g) ibeatgroup/ibeat_v2:release100 --t2 HF00093_T2_3x_acquisitions.nii.gz --age 44 --out_dir results --sub_name sub-001 ************************************************ **iBEAT V2.0 pipeline processing** ************************************************ Pipeline input parameters: --t2 'HF00093_T2_3x_acquisitions.nii.gz' --age '44' --out_dir 'results' --sub_name 'sub-001' -- License error!!! Did you get a valid license from http://www.ibeatcloud.com? If you have got the license, please put it to your data folder where the container can see! \t\tFor example, if you run "docker run ... -v /your_pc_path:/InfantData ..." \t\t\tthe license should be in the folder "your_pc_path". ubuntu@ip-172-31-3-79:~$ client_loop: send disconnect: Broken pipe (base) mathieuboudreau@Mathieus-MacBook-Pro Downloads % ssh -i "lowfieldinfant.pem" ubuntu@ec2-3-99-150-155.ca-central-1.compute.amazonaws.com ============================================================================= __| __|_ ) _| ( / Deep Learning AMI GPU PyTorch 1.12.0 (Ubuntu 20.04) ___|\___|___| ============================================================================= Welcome to Ubuntu 20.04.4 LTS (GNU/Linux 5.15.0-1017-aws x86_64v) * To activate pre-built pytorch environment, run: 'source activate pytorch' * To activate base conda environment upon login, run: 'conda config --set auto_activate_base true' * NVIDIA driver version: 510.73.08 * CUDA version: 11.6 AWS Deep Learning AMI Homepage: https://aws.amazon.com/machine-learning/amis/ Release Notes: https://docs.aws.amazon.com/dlami/latest/devguide/appendix-ami-release-notes.html Support: https://forums.aws.amazon.com/forum.jspa?forumID=263 For a fully managed experience, check out Amazon SageMaker at https://aws.amazon.com/sagemaker Security scan reports for python packages are located at: /opt/aws/dlami/info/ ============================================================================= * Documentation: https://help.ubuntu.com * Management: https://landscape.canonical.com * Support: https://ubuntu.com/advantage System information as of Fri Sep 9 18:30:28 UTC 2022 System load: 0.69 Processes: 184 Usage of /: 72.1% of 43.55GB Users logged in: 0 Memory usage: 0% IPv4 address for docker0: 172.17.0.1 Swap usage: 0% IPv4 address for ens3: 172.31.3.79 0 updates can be applied immediately. The list of available updates is more than a week old. To check for new updates run: sudo apt update New release '22.04.1 LTS' available. Run 'do-release-upgrade' to upgrade to it. 1 updates could not be installed automatically. For more details, see /var/log/unattended-upgrades/unattended-upgrades.log Last login: Fri Sep 9 17:14:16 2022 from 173.35.173.46 ubuntu@ip-172-31-3-79:~$ docker run --gpus=all --rm -it -v $(pwd)/data:/InfantData --user $(id -u):$(id -g) ibeatgroup/ibeat_v2:release100 --t2 HF00093_T2_3x_acquisitions.nii.gz --age 44 --out_dir results --sub_name sub-001 ************************************************ **iBEAT V2.0 pipeline processing** ************************************************ Pipeline input parameters: --t2 'HF00093_T2_3x_acquisitions.nii.gz' --age '44' --out_dir 'results' --sub_name 'sub-001' -- No T1w image inputed! T2w image: HF00093_T2_3x_acquisitions.nii.gz Subject age in months: 44 Model age in months: 18 Output folder: results/sub-001 Orientation code: T2w-LPI Doing reorientation and resampling... Resampling... Interpolator: Linear Direction: 1 0 0 0 1 0 0 0 1 Output Size: [175, 158, 288] Output Resolution: [0.8, 0.8, 0.8] Output Origin: [0, 0, 0] Original T2 resolution in RAI: 1.750000x1.800000x1.800000, new T2 resolution: 0.8x0.8x0.8 Original T2 size in RAI: 80x70x128, new T2 size: 175x158x288 Reorientation and sampling done! Doing n3 bias field correction... N3 bias field correction input parameters: -i 'results/sub-001/T2-RO-RS.nii.gz' -o 'results/sub-001/T2-RO-RS-n3.nii.gz' -- Input image: results/sub-001/T2-RO-RS.nii.gz Output image: results/sub-001/T2-RO-RS-n3.nii.gz Image size:175,158,288 -- size: 175,158,288, Res: 0.800000,0.800000,0.800000 N3 bias field correction done! Doing skull stripping... Matching Histogram... /usr/local/InfantPipeline/bin/Skullstripping_LiModel.py:38: SyntaxWarning: assertion is always true, perhaps remove parentheses? assert(dSeg[i]<=dFA[i], 'The output patch size should be smalled than input patch size!') WARNING: Logging before InitGoogleLogging() is written to STDERR W0909 18:32:18.923687 1837 _caffe.cpp:122] DEPRECATION WARNING - deprecated use of Python interface W0909 18:32:18.923735 1837 _caffe.cpp:123] Use this instead (with the named "weights" parameter): W0909 18:32:18.923751 1837 _caffe.cpp:125] Net('/usr/local/InfantPipeline/bin/LiModule/Skullstripping_T2.prototxt', 1, weights='/usr/local/InfantPipeline/bin/LiModule/Skullstripping_T2.caffemodel') W0909 18:32:18.932214 1837 upgrade_proto.cpp:71] Note that future Caffe releases will only support input layers and not input fields. In image: results/sub-001/T2-RO-RS-n3-histForSkull.nii.gz Out image: results/sub-001/T2-RO-RS-n3-SkullPM.nii.gz Model path: /usr/local/InfantPipeline/bin/LiModule/Skullstripping_T2.caffemodel Network structure path: /usr/local/InfantPipeline/bin/LiModule/Skullstripping_T2.prototxt ('Input patch size: ', [64, 64, 64]) ('Output patch size: ', [64, 64, 64]) ('Step size: ', [24, 24, 24]) F0909 18:32:25.907539 1837 im2col.cu:497] Check failed: error == cudaSuccess (209 vs. 0) no kernel image is available for execution on the device *** Check failure stack trace: *** /bin/edit_image: line 1954: 1837 Aborted (core dumped) CUDA_VISIBLE_DEVICES=${strGPUID} python ${strPipelineFolder}/Skullstripping_LiModel.py -i ${strT2FileRORSN3ForSkullStripping} -m ${strT2SkullStrippingModel} -p ${strT2SkullStrippingProto} -o ${strT2SkullStrippingPM} --size_in 64 64 64 --size_out 64 64 64 --size_step 24 24 24 ExceptionObject caught! itk::ImageFileReaderException (0x561b2598f1d0) Location: "void itk::ImageFileReader::GenerateOutputInformation() [with TOutputImage = itk::Image; ConvertPixelTraits = itk::DefaultConvertPixelTraits]" File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx Line: 143 Description: Could not create IO object for reading file results/sub-001/T2-RO-RS-n3-SkullPM.nii.gz The file doesn't exist. Filename = results/sub-001/T2-RO-RS-n3-SkullPM.nii.gz ExceptionObject caught! itk::ImageFileReaderException (0x5631c70f21d0) Location: "void itk::ImageFileReader::GenerateOutputInformation() [with TOutputImage = itk::Image; ConvertPixelTraits = itk::DefaultConvertPixelTraits]" File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx Line: 143 Description: Could not create IO object for reading file results/sub-001/T2-RO-RS-n3-SkullMask.nii.gz The file doesn't exist. Filename = results/sub-001/T2-RO-RS-n3-SkullMask.nii.gz ExceptionObject caught! itk::ImageFileReaderException (0x556cae4ab1d0) Location: "void itk::ImageFileReader::GenerateOutputInformation() [with TOutputImage = itk::Image; ConvertPixelTraits = itk::DefaultConvertPixelTraits]" File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx Line: 143 Description: Could not create IO object for reading file results/sub-001/T2-RO-RS-n3-SkullMask.nii.gz The file doesn't exist. Filename = results/sub-001/T2-RO-RS-n3-SkullMask.nii.gz Error! File results/sub-001/T2-RO-RS-n3-SkullMask.nii.gz not exist! ExceptionObject caught! itk::ImageFileReaderException (0x55f81f2931d0) Location: "void itk::ImageFileReader::GenerateOutputInformation() [with TOutputImage = itk::Image; ConvertPixelTraits = itk::DefaultConvertPixelTraits]" File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx Line: 143 Description: Could not create IO object for reading file results/sub-001/T2-RO-RS-n3-SkullMask.nii.gz The file doesn't exist. Filename = results/sub-001/T2-RO-RS-n3-SkullMask.nii.gz ExceptionObject caught! itk::ImageFileReaderException (0x561296054f60) Location: "void itk::ImageFileReader::GenerateOutputInformation() [with TOutputImage = itk::Image; ConvertPixelTraits = itk::DefaultConvertPixelTraits]" File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx Line: 143 Description: Could not create IO object for reading file results/sub-001/T2-RO-RS-n3-SkullMask.nii.gz The file doesn't exist. Filename = results/sub-001/T2-RO-RS-n3-SkullMask.nii.gz cp: cannot stat 'results/sub-001/T2-RO-RS-n3-SkullMask.nii.gz': No such file or directory Skull stripping done! Doing cerebellum removal... N3 bias field correction input parameters: -i 'results/sub-001/T2-RO-RS-n3-SkullStripped.nii.gz' -o 'results/sub-001/T2-RO-RS-n3-SkullStripped-n3.nii.gz' -- Input image: results/sub-001/T2-RO-RS-n3-SkullStripped.nii.gz Output image: results/sub-001/T2-RO-RS-n3-SkullStripped-n3.nii.gz ExceptionObject caught! itk::ImageFileReaderException (0x55ad767521d0) Location: "void itk::ImageFileReader::GenerateOutputInformation() [with TOutputImage = itk::Image; ConvertPixelTraits = itk::DefaultConvertPixelTraits]" File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx Line: 143 Description: Could not create IO object for reading file results/sub-001/T2-RO-RS-n3-SkullStripped.nii.gz The file doesn't exist. Filename = results/sub-001/T2-RO-RS-n3-SkullStripped.nii.gz Error! File results/sub-001/T2-RO-RS-n3-SkullStripped-scaled.nii.gz not exist! Error! File results/sub-001/T2-RO-RS-n3-SkullStripped-scaled.nii.gz not exist! -- size: Error! File results/sub-001/T2-RO-RS-n3-SkullStripped-scaled.nii.gz not exist!,Error! File results/sub-001/T2-RO-RS-n3-SkullStripped-scaled.nii.gz not exist!,Error! File results/sub-001/T2-RO-RS-n3-SkullStripped-scaled.nii.gz not exist!, Res: Error! File results/sub-001/T2-RO-RS-n3-SkullStripped-scaled.nii.gz not exist!,Error! File results/sub-001/T2-RO-RS-n3-SkullStripped-scaled.nii.gz not exist!,Error! File results/sub-001/T2-RO-RS-n3-SkullStripped-scaled.nii.gz not exist! Error open: results/sub-001/T2-RO-RS-n3-SkullStripped-scaled-forn3.img Error open: results/sub-001/T2-RO-RS-n3-SkullStripped-scaled-forn3-n3.img Error open: results/sub-001/T2-RO-RS-n3-SkullStripped-scaled-forn3-n3.img cp: cannot stat 'results/sub-001/T2-RO-RS-n3-SkullStripped-scaled-forn3.hdr': No such file or directory Error! File results/sub-001/T2-RO-RS-n3-SkullStripped-scaled-forn3-n3.hdr not exist! rm: cannot remove 'results/sub-001/T2-RO-RS-n3-SkullStripped-scaled.nii.gz': No such file or directory rm: cannot remove 'results/sub-001/T2-RO-RS-n3-SkullStripped-scaled-forn3.hdr': No such file or directory rm: cannot remove 'results/sub-001/T2-RO-RS-n3-SkullStripped-scaled-forn3.img': No such file or directory rm: cannot remove 'results/sub-001/T2-RO-RS-n3-SkullStripped-scaled-forn3-n3.hdr': No such file or directory rm: cannot remove 'results/sub-001/T2-RO-RS-n3-SkullStripped-scaled-forn3-n3.img': No such file or directory Image Exception : #22 :: ERROR: Could not open image results/sub-001/T2-RO-RS-n3-SkullStripped-n3 terminate called after throwing an instance of 'RBD_COMMON::BaseException' /bin/edit_image: line 1954: 1921 Aborted (core dumped) flirt -in ${strT2FileRORSN3SkullStrippedN3} -ref ${strT2CereFlirtTemplate} -out ${strT2FileRORSN3SkullStrippedN3ForCereFlirt} -omat ${strT2FileRORSN3SkullStrippedN3ForCereFlirtMat} ExceptionObject caught! itk::ImageFileReaderException (0x55e8ff92b1d0) Location: "void itk::ImageFileReader::GenerateOutputInformation() [with TOutputImage = itk::Image; ConvertPixelTraits = itk::DefaultConvertPixelTraits]" File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx Line: 143 Description: Could not create IO object for reading file results/sub-001/T2-RO-RS-n3-SkullStripped-n3-histForCere-flirt.nii.gz The file doesn't exist. Filename = results/sub-001/T2-RO-RS-n3-SkullStripped-n3-histForCere-flirt.nii.gz /usr/local/InfantPipeline/bin/convert_mat: line 14: results/sub-001/T2-RO-RS-n3-SkullStripped-n3-histForCere-flirt.mat: No such file or directory Cannot read input-matrix /usr/local/InfantPipeline/bin/CerebellumRemoval_LiModel.py:33: SyntaxWarning: assertion is always true, perhaps remove parentheses? assert(dSeg[i]<=dFA[i], 'The output patch size should be smalled than input patch size!') WARNING: Logging before InitGoogleLogging() is written to STDERR W0909 18:32:29.091787 1930 _caffe.cpp:122] DEPRECATION WARNING - deprecated use of Python interface W0909 18:32:29.091840 1930 _caffe.cpp:123] Use this instead (with the named "weights" parameter): W0909 18:32:29.091846 1930 _caffe.cpp:125] Net('/usr/local/InfantPipeline/bin/LiModule/CerebellumRemoval_T2.prototxt', 1, weights='/usr/local/InfantPipeline/bin/LiModule/CerebellumRemoval_T2.caffemodel') W0909 18:32:29.096426 1930 upgrade_proto.cpp:71] Note that future Caffe releases will only support input layers and not input fields. Traceback (most recent call last): File "/usr/local/InfantPipeline/bin/CerebellumRemoval_LiModel.py", line 184, in main(sys.argv[1:]) File "/usr/local/InfantPipeline/bin/CerebellumRemoval_LiModel.py", line 151, in main imgOrg=sitk.ReadImage(strInImage) File "/usr/local/lib/python2.7/dist-packages/SimpleITK/SimpleITK.py", line 8876, in ReadImage return _SimpleITK.ReadImage(*args) RuntimeError: Exception thrown in SimpleITK ReadImage: /tmp/SimpleITK/Code/IO/src/sitkImageReaderBase.cxx:99: sitk::ERROR: The file "results/sub-001/T2-RO-RS-n3-SkullStripped-n3-histForCere-flirt-hist.nii.gz" does not exist. Image Exception : #22 :: ERROR: Could not open image results/sub-001/T2-RO-RS-n3-SkullStripped-n3 terminate called after throwing an instance of 'RBD_COMMON::BaseException' /bin/edit_image: line 1954: 1953 Aborted (core dumped) flirt -in ${strT2CerebralPMByFlirt} -ref ${strT2FileRORSN3SkullStrippedN3} -out ${strT2CerebralPM} -applyxfm -init ${strT2FileRORSN3SkullStrippedN3ForCereFlirtMatInverse} Image Exception : #22 :: ERROR: Could not open image results/sub-001/T2-RO-RS-n3-SkullStripped-n3 terminate called after throwing an instance of 'RBD_COMMON::BaseException' /bin/edit_image: line 1954: 1954 Aborted (core dumped) flirt -in ${strT2CerebellumPMByFlirt} -ref ${strT2FileRORSN3SkullStrippedN3} -out ${strT2CerebellumPM} -applyxfm -init ${strT2FileRORSN3SkullStrippedN3ForCereFlirtMatInverse} ExceptionObject caught! itk::ImageFileReaderException (0x558a8df491d0) Location: "void itk::ImageFileReader::GenerateOutputInformation() [with TOutputImage = itk::Image; ConvertPixelTraits = itk::DefaultConvertPixelTraits]" File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx Line: 143 Description: Could not create IO object for reading file results/sub-001/T2-RO-RS-n3-SkullStripped-n3-CerebellumPM.nii.gz The file doesn't exist. Filename = results/sub-001/T2-RO-RS-n3-SkullStripped-n3-CerebellumPM.nii.gz ExceptionObject caught! itk::ImageFileReaderException (0x55a157ad41d0) Location: "void itk::ImageFileReader::GenerateOutputInformation() [with TOutputImage = itk::Image; ConvertPixelTraits = itk::DefaultConvertPixelTraits]" File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx Line: 143 Description: Could not create IO object for reading file results/sub-001/T2-RO-RS-n3-SkullStripped-n3-CerebellumLabel.nii.gz The file doesn't exist. Filename = results/sub-001/T2-RO-RS-n3-SkullStripped-n3-CerebellumLabel.nii.gz /bin/edit_image: line 1732: /2: syntax error: operand expected (error token is "/2") Fatal error: Segmentation image cannot be found! ubuntu@ip-172-31-3-79:~$ cuda cuda-gdb cuda-gdbserver cuda-memcheck cuda-uninstaller cudafe++ ubuntu@ip-172-31-3-79:~$ nvidia- nvidia-bug-report.sh nvidia-container-toolkit nvidia-debugdump nvidia-ngx-updater nvidia-sleep.sh nvidia-container-cli nvidia-ctk nvidia-docker nvidia-persistenced nvidia-smi nvidia-container-runtime nvidia-cuda-mps-control nvidia-installer nvidia-powerd nvidia-uninstall nvidia-container-runtime-hook nvidia-cuda-mps-server nvidia-modprobe nvidia-settings nvidia-xconfig ubuntu@ip-172-31-3-79:~$ nvidia-smi Fri Sep 9 18:35:23 2022 +-----------------------------------------------------------------------------+ | NVIDIA-SMI 510.73.08 Driver Version: 510.73.08 CUDA Version: 11.6 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 Tesla V100-SXM2... On | 00000000:00:1E.0 Off | 0 | | N/A 39C P0 24W / 300W | 0MiB / 16384MiB | 0% Default | | | | N/A | +-------------------------------+----------------------+----------------------+ +-----------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=============================================================================| | No running processes found | +-----------------------------------------------------------------------------+ ubuntu@ip-172-31-3-79:~$ docker docker docker-init docker-proxy dockerd dockerd-rootless-setuptool.sh dockerd-rootless.sh ubuntu@ip-172-31-3-79:~$ ```

[ajdneuro]

I am also getting this error above

[Zhengwang-Wu]

@mathieuboudreau @ajdneuro We have fixed the issue. You can have a test using latest version: docker pull ibeatgroup/ibeat_v2:release120

[mblesac]

Hi @Zhengwang-Wu,

I'm getting the same error but with the latest version (release120):

sudo docker run --gpus=all --rm -it -v /EBC/home/mblesa/ibeat2_test:/InfantData ibeatgroup/ibeat_v2:release120 --t1 8018-T1.nii --t2 8018-T2.nii --age 0 --out_dir /EBC/home/mblesa/ibeat2_test --sub_name 8018

************************************************
**iBEAT V2.0 pipeline processing**
************************************************
Pipeline input parameters:  --t1 '8018-T1.nii' --t2 '8018-T2.nii' --age '0' --out_dir '/EBC/home/mblesa/ibeat2_test' --sub_name '8018' --
Ceretype: 1
T1w image: 8018-T1.nii
T2w image: 8018-T2.nii
Subject age in months: 0
Model age in months: 0
Output folder: /EBC/home/mblesa/ibeat2_test/8018

Orientation code: T1w-LPI T2w-LPI
Doing reorientation and resampling...
Original T1w resolution in RAI: 1.000004x1.000000x1.000000. New T1w resolution: 0.8x0.8x0.8
Original T1w size in RAI: 160x160x160. New T1w size: 200x200x200
Original T2w resolution in RAI: 0.999997x1.000000x1.000000. New T2w resolution: 0.8x0.8x0.8
Original T2w size in RAI: 160x192x128. New T2w size: 200x240x160
Reorientation and resampling done!
Doing N3 bias field correction...
N3 bias field correction input parameters:  -i '/EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS.nii.gz' -o '/EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3.nii.gz' --
Input image: /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS.nii.gz
Output image: /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3.nii.gz
Image size:200,200,200 -- size: 200,200,200, Res: 0.800000,0.800000,0.800000
N3 bias field correction input parameters:  -i '/EBC/home/mblesa/ibeat2_test/8018/T2-RO-RS.nii.gz' -o '/EBC/home/mblesa/ibeat2_test/8018/T2-RO-RS-n3.nii.gz' --
Input image: /EBC/home/mblesa/ibeat2_test/8018/T2-RO-RS.nii.gz
Output image: /EBC/home/mblesa/ibeat2_test/8018/T2-RO-RS-n3.nii.gz
Image size:200,240,160 -- size: 200,240,160, Res: 0.800000,0.800000,0.800000
N3 bias field correction done!
Doing skull stripping...
/usr/local/InfantPipeline/bin/Skullstripping_LiModel.py:38: SyntaxWarning: assertion is always true, perhaps remove parentheses?
  assert(dSeg[i]<=dFA[i], 'The output patch size should be smalled than input patch size!')
WARNING: Logging before InitGoogleLogging() is written to STDERR
W0405 10:53:52.245676  2850 _caffe.cpp:122] DEPRECATION WARNING - deprecated use of Python interface
W0405 10:53:52.245741  2850 _caffe.cpp:123] Use this instead (with the named "weights" parameter):
W0405 10:53:52.245751  2850 _caffe.cpp:125] Net('/usr/local/InfantPipeline/bin/LiModule/Skullstripping_T1.prototxt', 1, weights='/usr/local/InfantPipeline/bin/LiModule/Skullstripping_T1.caffemodel')
W0405 10:53:52.272204  2850 upgrade_proto.cpp:71] Note that future Caffe releases will only support input layers and not input fields.
In image: /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-histForSkull.nii.gz
Out image: /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullPM.nii.gz
Model path: /usr/local/InfantPipeline/bin/LiModule/Skullstripping_T1.caffemodel
Network structure path: /usr/local/InfantPipeline/bin/LiModule/Skullstripping_T1.prototxt
('Input patch size:  ', [64, 64, 64])
('Output patch size: ', [64, 64, 64])
('Step size: ', [12, 12, 12])
F0405 10:54:05.781658  2850 im2col.cu:497] Check failed: error == cudaSuccess (209 vs. 0)  no kernel image is available for execution on the device
*** Check failure stack trace: ***
/bin/edit_image: line 2005:  2850 Aborted                 (core dumped) CUDA_VISIBLE_DEVICES=${strGPUID} python ${strPipelineFolder}/Skullstripping_LiModel.py -i ${strT1FileRORSN3ForSkullStripping} -m ${strT1SkullStrippingModel} -p ${strT1SkullStrippingProto} -o ${strT1SkullStrippingPM} --size_in 64 64 64 --size_out 64 64 64 --size_step 12 12 12
ExceptionObject caught!

itk::ImageFileReaderException (0x55c48e1001d0)
Location: "void itk::ImageFileReader<TOutputImage, ConvertPixelTraits>::GenerateOutputInformation() [with TOutputImage = itk::Image<double, 3>; ConvertPixelTraits = itk::DefaultConvertPixelTraits<double>]"
File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx
Line: 143
Description:  Could not create IO object for reading file /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullPM.nii.gz
The file doesn't exist.
Filename = /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullPM.nii.gz

ExceptionObject caught!

itk::ImageFileReaderException (0x55822f8621d0)
Location: "void itk::ImageFileReader<TOutputImage, ConvertPixelTraits>::GenerateOutputInformation() [with TOutputImage = itk::Image<double, 3>; ConvertPixelTraits = itk::DefaultConvertPixelTraits<double>]"
File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx
Line: 143
Description:  Could not create IO object for reading file /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullMask.nii.gz
The file doesn't exist.
Filename = /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullMask.nii.gz

ExceptionObject caught!

itk::ImageFileReaderException (0x55a0151811d0)
Location: "void itk::ImageFileReader<TOutputImage, ConvertPixelTraits>::GenerateOutputInformation() [with TOutputImage = itk::Image<double, 3>; ConvertPixelTraits = itk::DefaultConvertPixelTraits<double>]"
File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx
Line: 143
Description:  Could not create IO object for reading file /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullMask.nii.gz
The file doesn't exist.
Filename = /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullMask.nii.gz

ExceptionObject caught!

itk::ImageFileReaderException (0x55fb24bc01d0)
Location: "void itk::ImageFileReader<TOutputImage, ConvertPixelTraits>::GenerateOutputInformation() [with TOutputImage = itk::Image<double, 3>; ConvertPixelTraits = itk::DefaultConvertPixelTraits<double>]"
File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx
Line: 143
Description:  Could not create IO object for reading file /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullMask.nii.gz
The file doesn't exist.
Filename = /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullMask.nii.gz

ExceptionObject caught!

itk::ImageFileReaderException (0x55e54e623db0)
Location: "void itk::ImageFileReader<TOutputImage, ConvertPixelTraits>::GenerateOutputInformation() [with TOutputImage = itk::Image<double, 3>; ConvertPixelTraits = itk::DefaultConvertPixelTraits<double>]"
File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx
Line: 143
Description:  Could not create IO object for reading file /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullMask.nii.gz
The file doesn't exist.
Filename = /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullMask.nii.gz

Matching Histogram...
/usr/local/InfantPipeline/bin/Skullstripping_LiModel.py:38: SyntaxWarning: assertion is always true, perhaps remove parentheses?
  assert(dSeg[i]<=dFA[i], 'The output patch size should be smalled than input patch size!')
WARNING: Logging before InitGoogleLogging() is written to STDERR
W0405 10:54:12.640048  3102 _caffe.cpp:122] DEPRECATION WARNING - deprecated use of Python interface
W0405 10:54:12.640084  3102 _caffe.cpp:123] Use this instead (with the named "weights" parameter):
W0405 10:54:12.640105  3102 _caffe.cpp:125] Net('/usr/local/InfantPipeline/bin/LiModule/Skullstripping_T2.prototxt', 1, weights='/usr/local/InfantPipeline/bin/LiModule/Skullstripping_T2.caffemodel')
W0405 10:54:12.663651  3102 upgrade_proto.cpp:71] Note that future Caffe releases will only support input layers and not input fields.
F0405 10:54:22.874893  3102 im2col.cu:497] Check failed: error == cudaSuccess (209 vs. 0)  no kernel image is available for execution on the device
*** Check failure stack trace: ***
/bin/edit_image: line 2005:  3102 Aborted                 (core dumped) CUDA_VISIBLE_DEVICES=${strGPUID} python ${strPipelineFolder}/Skullstripping_LiModel.py -i ${strT2FileRORSN3ForSkullStripping} -m ${strT2SkullStrippingModel} -p ${strT2SkullStrippingProto} -o ${strT2SkullStrippingPM} --size_in 64 64 64 --size_out 64 64 64 --size_step 16 16 16 > /dev/null
ExceptionObject caught!

itk::ImageFileReaderException (0x55ca8ee241d0)
Location: "void itk::ImageFileReader<TOutputImage, ConvertPixelTraits>::GenerateOutputInformation() [with TOutputImage = itk::Image<double, 3>; ConvertPixelTraits = itk::DefaultConvertPixelTraits<double>]"
File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx
Line: 143
Description:  Could not create IO object for reading file /EBC/home/mblesa/ibeat2_test/8018/T2-RO-RS-n3-SkullPM.nii.gz
The file doesn't exist.
Filename = /EBC/home/mblesa/ibeat2_test/8018/T2-RO-RS-n3-SkullPM.nii.gz

ExceptionObject caught!

itk::ImageFileReaderException (0x555f517a91d0)
Location: "void itk::ImageFileReader<TOutputImage, ConvertPixelTraits>::GenerateOutputInformation() [with TOutputImage = itk::Image<double, 3>; ConvertPixelTraits = itk::DefaultConvertPixelTraits<double>]"
File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx
Line: 143
Description:  Could not create IO object for reading file /EBC/home/mblesa/ibeat2_test/8018/T2-RO-RS-n3-SkullMask.nii.gz
The file doesn't exist.
Filename = /EBC/home/mblesa/ibeat2_test/8018/T2-RO-RS-n3-SkullMask.nii.gz

ExceptionObject caught!

itk::ImageFileReaderException (0x555c455d51d0)
Location: "void itk::ImageFileReader<TOutputImage, ConvertPixelTraits>::GenerateOutputInformation() [with TOutputImage = itk::Image<double, 3>; ConvertPixelTraits = itk::DefaultConvertPixelTraits<double>]"
File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx
Line: 143
Description:  Could not create IO object for reading file /EBC/home/mblesa/ibeat2_test/8018/T2-RO-RS-n3-SkullMask.nii.gz
The file doesn't exist.
Filename = /EBC/home/mblesa/ibeat2_test/8018/T2-RO-RS-n3-SkullMask.nii.gz

Error! File /EBC/home/mblesa/ibeat2_test/8018/T2-RO-RS-n3-SkullMask.nii.gz not exist!

ExceptionObject caught!

itk::ImageFileReaderException (0x560550eb61d0)
Location: "void itk::ImageFileReader<TOutputImage, ConvertPixelTraits>::GenerateOutputInformation() [with TOutputImage = itk::Image<double, 3>; ConvertPixelTraits = itk::DefaultConvertPixelTraits<double>]"
File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx
Line: 143
Description:  Could not create IO object for reading file /EBC/home/mblesa/ibeat2_test/8018/T2-RO-RS-n3-SkullMask.nii.gz
The file doesn't exist.
Filename = /EBC/home/mblesa/ibeat2_test/8018/T2-RO-RS-n3-SkullMask.nii.gz

ExceptionObject caught!

itk::ImageFileReaderException (0x563d1cebbdb0)
Location: "void itk::ImageFileReader<TOutputImage, ConvertPixelTraits>::GenerateOutputInformation() [with TOutputImage = itk::Image<double, 3>; ConvertPixelTraits = itk::DefaultConvertPixelTraits<double>]"
File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx
Line: 143
Description:  Could not create IO object for reading file /EBC/home/mblesa/ibeat2_test/8018/T2-RO-RS-n3-SkullMask.nii.gz
The file doesn't exist.
Filename = /EBC/home/mblesa/ibeat2_test/8018/T2-RO-RS-n3-SkullMask.nii.gz

Skull stripping done!
Doing T1 T2 alignment...
cp: cannot stat '/EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullMask.nii.gz': No such file or directory
cp: cannot stat '/EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped.nii.gz': No such file or directory
Error!!! Input T1 file /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped.nii.gz not exist!
ExceptionObject caught!

itk::ImageFileReaderException (0x5578947f6db0)
Location: "void itk::ImageFileReader<TOutputImage, ConvertPixelTraits>::GenerateOutputInformation() [with TOutputImage = itk::Image<double, 3>; ConvertPixelTraits = itk::DefaultConvertPixelTraits<double>]"
File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx
Line: 143
Description:  Could not create IO object for reading file /EBC/home/mblesa/ibeat2_test/8018/brain_mask.nii.gz
The file doesn't exist.
Filename = /EBC/home/mblesa/ibeat2_test/8018/brain_mask.nii.gz

T1 T2 Alignment done!
Doing cerebellum removal...
N3 bias field correction input parameters:  -i '/EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped.nii.gz' -o '/EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-n3.nii.gz' --
Input image: /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped.nii.gz
Output image: /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-n3.nii.gz
ExceptionObject caught!

itk::ImageFileReaderException (0x56161771e1d0)
Location: "void itk::ImageFileReader<TOutputImage, ConvertPixelTraits>::GenerateOutputInformation() [with TOutputImage = itk::Image<double, 3>; ConvertPixelTraits = itk::DefaultConvertPixelTraits<double>]"
File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx
Line: 143
Description:  Could not create IO object for reading file /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped.nii.gz
The file doesn't exist.
Filename = /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped.nii.gz

Error! File /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-scaled.nii.gz not exist!

Error! File /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-scaled.nii.gz not exist! -- size: Error! File /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-scaled.nii.gz not exist!,Error! File /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-scaled.nii.gz not exist!,Error! File /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-scaled.nii.gz not exist!, Res: Error! File /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-scaled.nii.gz not exist!,Error! File /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-scaled.nii.gz not exist!,Error! File /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-scaled.nii.gz not exist!

Error open: /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-scaled-forn3.img

Error open: /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-scaled-forn3-n3.img

Error open: /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-scaled-forn3-n3.img

cp: cannot stat '/EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-scaled-forn3.hdr': No such file or directory
Error! File /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-scaled-forn3-n3.hdr not exist!

rm: cannot remove '/EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-scaled.nii.gz': No such file or directory
rm: cannot remove '/EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-scaled-forn3.hdr': No such file or directory
rm: cannot remove '/EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-scaled-forn3.img': No such file or directory
rm: cannot remove '/EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-scaled-forn3-n3.hdr': No such file or directory
rm: cannot remove '/EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-scaled-forn3-n3.img': No such file or directory
ExceptionObject caught!

itk::ImageFileReaderException (0x556aee1c91d0)
Location: "void itk::ImageFileReader<TOutputImage, ConvertPixelTraits>::GenerateOutputInformation() [with TOutputImage = itk::Image<double, 3>; ConvertPixelTraits = itk::DefaultConvertPixelTraits<double>]"
File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx
Line: 143
Description:  Could not create IO object for reading file /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-n3.nii.gz
The file doesn't exist.
Filename = /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-n3.nii.gz

Image Exception : #22 :: ERROR: Could not open image /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-n3-histForCere
terminate called after throwing an instance of 'RBD_COMMON::BaseException'
/bin/edit_image: line 2005:  3363 Aborted                 (core dumped) flirt -in ${strT1FileRORSN3SkullStrippedN3ForCere} -ref ${strT1CereHistTemplate} -out ${strT1FileRORSN3SkullStrippedN3ForCereFlirt} -omat ${strT1FileRORSN3SkullStrippedN3ForCereFlirtMat}
ExceptionObject caught!

itk::ImageFileReaderException (0x55cd8febf1d0)
Location: "void itk::ImageFileReader<TOutputImage, ConvertPixelTraits>::GenerateOutputInformation() [with TOutputImage = itk::Image<double, 3>; ConvertPixelTraits = itk::DefaultConvertPixelTraits<double>]"
File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx
Line: 143
Description:  Could not create IO object for reading file /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-n3-histForCere-flirt.nii.gz
The file doesn't exist.
Filename = /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-n3-histForCere-flirt.nii.gz

/usr/local/InfantPipeline/bin/convert_mat: line 14: /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-n3-histForCere-flirt.mat: No such file or directory
Cannot read input-matrix
/usr/local/InfantPipeline/bin/CerebellumRemoval_LiModel.py:33: SyntaxWarning: assertion is always true, perhaps remove parentheses?
  assert(dSeg[i]<=dFA[i], 'The output patch size should be smalled than input patch size!')
WARNING: Logging before InitGoogleLogging() is written to STDERR
W0405 10:54:27.540024  3372 _caffe.cpp:122] DEPRECATION WARNING - deprecated use of Python interface
W0405 10:54:27.540069  3372 _caffe.cpp:123] Use this instead (with the named "weights" parameter):
W0405 10:54:27.540073  3372 _caffe.cpp:125] Net('/usr/local/InfantPipeline/bin/LiModule/CerebellumRemoval_T1.prototxt', 1, weights='/usr/local/InfantPipeline/bin/LiModule/CerebellumRemoval_T1.caffemodel')
W0405 10:54:27.561908  3372 upgrade_proto.cpp:71] Note that future Caffe releases will only support input layers and not input fields.
Traceback (most recent call last):
  File "/usr/local/InfantPipeline/bin/CerebellumRemoval_LiModel.py", line 184, in <module>
    main(sys.argv[1:])
  File "/usr/local/InfantPipeline/bin/CerebellumRemoval_LiModel.py", line 151, in main
    imgOrg=sitk.ReadImage(strInImage)
  File "/usr/local/lib/python2.7/dist-packages/SimpleITK/SimpleITK.py", line 8876, in ReadImage
    return _SimpleITK.ReadImage(*args)
RuntimeError: Exception thrown in SimpleITK ReadImage: /tmp/SimpleITK/Code/IO/src/sitkImageReaderBase.cxx:99:
sitk::ERROR: The file "/EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-n3-histForCere-flirt-hist.nii.gz" does not exist.
Image Exception : #22 :: ERROR: Could not open image /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-n3-histForCere
terminate called after throwing an instance of 'RBD_COMMON::BaseException'
/bin/edit_image: line 2005:  3505 Aborted                 (core dumped) flirt -in ${strT1CerebralPMByFlirt} -ref ${strT1FileRORSN3SkullStrippedN3ForCere} -out ${strT1CerebralPM} -applyxfm -init ${strT1FileRORSN3SkullStrippedN3ForCereFlirtMatInverse}
Image Exception : #22 :: ERROR: Could not open image /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-n3-histForCere
terminate called after throwing an instance of 'RBD_COMMON::BaseException'
/bin/edit_image: line 2005:  3506 Aborted                 (core dumped) flirt -in ${strT1CerebellumPMByFlirt} -ref ${strT1FileRORSN3SkullStrippedN3ForCere} -out ${strT1CerebellumPM} -applyxfm -init ${strT1FileRORSN3SkullStrippedN3ForCereFlirtMatInverse}
ExceptionObject caught!

itk::ImageFileReaderException (0x55d0ef62c1d0)
Location: "void itk::ImageFileReader<TOutputImage, ConvertPixelTraits>::GenerateOutputInformation() [with TOutputImage = itk::Image<double, 3>; ConvertPixelTraits = itk::DefaultConvertPixelTraits<double>]"
File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx
Line: 143
Description:  Could not create IO object for reading file /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-n3-CerebellumPM.nii.gz
The file doesn't exist.
Filename = /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-n3-CerebellumPM.nii.gz

ExceptionObject caught!

itk::ImageFileReaderException (0x5654cbe981d0)
Location: "void itk::ImageFileReader<TOutputImage, ConvertPixelTraits>::GenerateOutputInformation() [with TOutputImage = itk::Image<double, 3>; ConvertPixelTraits = itk::DefaultConvertPixelTraits<double>]"
File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx
Line: 143
Description:  Could not create IO object for reading file /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-n3-CerebellumLabel.nii.gz
The file doesn't exist.
Filename = /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-n3-CerebellumLabel.nii.gz

/bin/edit_image: line 654: /2: syntax error: operand expected (error token is "/2")
Fatal error: Segmentation image cannot be found!
s1474782@EBC33:/EBC/home/mblesa/ibeat2_test$ docker run --rm --gpus=all nvidia/cuda:9.0-base nvidia-smi
Wed Apr  5 10:57:53 2023
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 450.51.06    Driver Version: 450.51.06    CUDA Version: 11.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla K80           On   | 00000000:84:00.0 Off |                    0 |
| N/A   32C    P8    26W / 149W |      0MiB / 11441MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   1  Tesla K80           On   | 00000000:85:00.0 Off |                    0 |
| N/A   27C    P8    30W / 149W |      0MiB / 11441MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

Any advice?

Best regards,

Manuel

[Zhengwang-Wu]

Hi @Zhengwang-Wu,

I'm getting the same error but with the latest version (release120):

Hi, @mblesac , it seems the Tesla K80 is a very old GPU, which has been deprecated from CUDA 11. If you have a relatively new GPU, it would be great if you can try it on the newer GPU. In the meantime, I will create a compatible version to accommodate K80.

[mblesac]

Hi @Zhengwang-Wu,

Thanks for your reply. Unfortunately, I don't have a newer GPU. I'll wait for the compatible version. Thanks in advance.

Best regards,

Manuel

[Zhengwang-Wu]

Hi, @mblesac , can you have a try of the version release110 (docker pull ibeatgroup/ibeat_v2:release110) to see whether it works on Tesla K80.

[mblesac]

Hi,

It doesn't work, I think is still the same error:

sudo docker run --gpus=all --rm -it -v /EBC/home/mblesa/ibeat2_test:/InfantData ibeatgroup/ibeat_v2:release110 --t1 8018-T1.nii --t2 8018-T2.nii --age 0 --out_dir /EBC/home/mblesa/ibeat2_test --sub_name 8018

************************************************
**iBEAT V2.0 pipeline processing**
************************************************
Pipeline input parameters:  --t1 '8018-T1.nii' --t2 '8018-T2.nii' --age '0' --out_dir '/EBC/home/mblesa/ibeat2_test' --sub_name '8018' --
Ceretype: 1
T1w image: 8018-T1.nii
T2w image: 8018-T2.nii
Subject age in months: 0
Model age in months: 0
Output folder: /EBC/home/mblesa/ibeat2_test/8018

Orientation code: T1w-LPI T2w-LPI
Doing reorientation and resampling...
Original T1w resolution in RAI: 1.000004x1.000000x1.000000. New T1w resolution: 0.8x0.8x0.8
Original T1w size in RAI: 160x160x160. New T1w size: 200x200x200
Original T2w resolution in RAI: 0.999997x1.000000x1.000000. New T2w resolution: 0.8x0.8x0.8
Original T2w size in RAI: 160x192x128. New T2w size: 200x240x160
Reorientation and resampling done!
Doing N3 bias field correction...
N3 bias field correction input parameters:  -i '/EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS.nii.gz' -o '/EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3.nii.gz' --
Input image: /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS.nii.gz
Output image: /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3.nii.gz
Image size:200,200,200 -- size: 200,200,200, Res: 0.800000,0.800000,0.800000
N3 bias field correction input parameters:  -i '/EBC/home/mblesa/ibeat2_test/8018/T2-RO-RS.nii.gz' -o '/EBC/home/mblesa/ibeat2_test/8018/T2-RO-RS-n3.nii.gz' --
Input image: /EBC/home/mblesa/ibeat2_test/8018/T2-RO-RS.nii.gz
Output image: /EBC/home/mblesa/ibeat2_test/8018/T2-RO-RS-n3.nii.gz
Image size:200,240,160 -- size: 200,240,160, Res: 0.800000,0.800000,0.800000
N3 bias field correction done!
Doing skull stripping...
/usr/local/InfantPipeline/bin/Skullstripping_LiModel.py:38: SyntaxWarning: assertion is always true, perhaps remove parentheses?
  assert(dSeg[i]<=dFA[i], 'The output patch size should be smalled than input patch size!')
WARNING: Logging before InitGoogleLogging() is written to STDERR
W0406 09:05:03.750039  2849 _caffe.cpp:122] DEPRECATION WARNING - deprecated use of Python interface
W0406 09:05:03.750114  2849 _caffe.cpp:123] Use this instead (with the named "weights" parameter):
W0406 09:05:03.750119  2849 _caffe.cpp:125] Net('/usr/local/InfantPipeline/bin/LiModule/Skullstripping_T1.prototxt', 1, weights='/usr/local/InfantPipeline/bin/LiModule/Skullstripping_T1.caffemodel')
W0406 09:05:03.756968  2849 upgrade_proto.cpp:71] Note that future Caffe releases will only support input layers and not input fields.
In image: /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-histForSkull.nii.gz
Out image: /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullPM.nii.gz
Model path: /usr/local/InfantPipeline/bin/LiModule/Skullstripping_T1.caffemodel
Network structure path: /usr/local/InfantPipeline/bin/LiModule/Skullstripping_T1.prototxt
('Input patch size:  ', [64, 64, 64])
('Output patch size: ', [64, 64, 64])
('Step size: ', [12, 12, 12])
F0406 09:05:08.497056  2849 im2col.cu:497] Check failed: error == cudaSuccess (209 vs. 0)  no kernel image is available for execution on the device
*** Check failure stack trace: ***
/bin/edit_image: line 2005:  2849 Aborted                 (core dumped) CUDA_VISIBLE_DEVICES=${strGPUID} python ${strPipelineFolder}/Skullstripping_LiModel.py -i ${strT1FileRORSN3ForSkullStripping} -m ${strT1SkullStrippingModel} -p ${strT1SkullStrippingProto} -o ${strT1SkullStrippingPM} --size_in 64 64 64 --size_out 64 64 64 --size_step 12 12 12
ExceptionObject caught!

itk::ImageFileReaderException (0x55e589cd71d0)
Location: "void itk::ImageFileReader<TOutputImage, ConvertPixelTraits>::GenerateOutputInformation() [with TOutputImage = itk::Image<double, 3>; ConvertPixelTraits = itk::DefaultConvertPixelTraits<double>]"
File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx
Line: 143
Description:  Could not create IO object for reading file /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullPM.nii.gz
The file doesn't exist.
Filename = /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullPM.nii.gz

ExceptionObject caught!

itk::ImageFileReaderException (0x560cc6e4e1d0)
Location: "void itk::ImageFileReader<TOutputImage, ConvertPixelTraits>::GenerateOutputInformation() [with TOutputImage = itk::Image<double, 3>; ConvertPixelTraits = itk::DefaultConvertPixelTraits<double>]"
File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx
Line: 143
Description:  Could not create IO object for reading file /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullMask.nii.gz
The file doesn't exist.
Filename = /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullMask.nii.gz

ExceptionObject caught!

itk::ImageFileReaderException (0x5591a2ae71d0)
Location: "void itk::ImageFileReader<TOutputImage, ConvertPixelTraits>::GenerateOutputInformation() [with TOutputImage = itk::Image<double, 3>; ConvertPixelTraits = itk::DefaultConvertPixelTraits<double>]"
File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx
Line: 143
Description:  Could not create IO object for reading file /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullMask.nii.gz
The file doesn't exist.
Filename = /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullMask.nii.gz

ExceptionObject caught!

itk::ImageFileReaderException (0x55d4aea9d1d0)
Location: "void itk::ImageFileReader<TOutputImage, ConvertPixelTraits>::GenerateOutputInformation() [with TOutputImage = itk::Image<double, 3>; ConvertPixelTraits = itk::DefaultConvertPixelTraits<double>]"
File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx
Line: 143
Description:  Could not create IO object for reading file /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullMask.nii.gz
The file doesn't exist.
Filename = /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullMask.nii.gz

ExceptionObject caught!

itk::ImageFileReaderException (0x56271f482db0)
Location: "void itk::ImageFileReader<TOutputImage, ConvertPixelTraits>::GenerateOutputInformation() [with TOutputImage = itk::Image<double, 3>; ConvertPixelTraits = itk::DefaultConvertPixelTraits<double>]"
File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx
Line: 143
Description:  Could not create IO object for reading file /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullMask.nii.gz
The file doesn't exist.
Filename = /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullMask.nii.gz

rm: cannot remove '/EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullPM.nii.gz': No such file or directory
Matching Histogram...
/usr/local/InfantPipeline/bin/Skullstripping_LiModel.py:38: SyntaxWarning: assertion is always true, perhaps remove parentheses?
  assert(dSeg[i]<=dFA[i], 'The output patch size should be smalled than input patch size!')
WARNING: Logging before InitGoogleLogging() is written to STDERR
W0406 09:05:15.047729  3103 _caffe.cpp:122] DEPRECATION WARNING - deprecated use of Python interface
W0406 09:05:15.047760  3103 _caffe.cpp:123] Use this instead (with the named "weights" parameter):
W0406 09:05:15.047782  3103 _caffe.cpp:125] Net('/usr/local/InfantPipeline/bin/LiModule/Skullstripping_T2.prototxt', 1, weights='/usr/local/InfantPipeline/bin/LiModule/Skullstripping_T2.caffemodel')
W0406 09:05:15.054427  3103 upgrade_proto.cpp:71] Note that future Caffe releases will only support input layers and not input fields.
F0406 09:05:19.525235  3103 im2col.cu:497] Check failed: error == cudaSuccess (209 vs. 0)  no kernel image is available for execution on the device
*** Check failure stack trace: ***
/bin/edit_image: line 2005:  3103 Aborted                 (core dumped) CUDA_VISIBLE_DEVICES=${strGPUID} python ${strPipelineFolder}/Skullstripping_LiModel.py -i ${strT2FileRORSN3ForSkullStripping} -m ${strT2SkullStrippingModel} -p ${strT2SkullStrippingProto} -o ${strT2SkullStrippingPM} --size_in 64 64 64 --size_out 64 64 64 --size_step 16 16 16 > /dev/null
ExceptionObject caught!

itk::ImageFileReaderException (0x5642df6c51d0)
Location: "void itk::ImageFileReader<TOutputImage, ConvertPixelTraits>::GenerateOutputInformation() [with TOutputImage = itk::Image<double, 3>; ConvertPixelTraits = itk::DefaultConvertPixelTraits<double>]"
File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx
Line: 143
Description:  Could not create IO object for reading file /EBC/home/mblesa/ibeat2_test/8018/T2-RO-RS-n3-SkullPM.nii.gz
The file doesn't exist.
Filename = /EBC/home/mblesa/ibeat2_test/8018/T2-RO-RS-n3-SkullPM.nii.gz

ExceptionObject caught!

itk::ImageFileReaderException (0x558e3daad1d0)
Location: "void itk::ImageFileReader<TOutputImage, ConvertPixelTraits>::GenerateOutputInformation() [with TOutputImage = itk::Image<double, 3>; ConvertPixelTraits = itk::DefaultConvertPixelTraits<double>]"
File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx
Line: 143
Description:  Could not create IO object for reading file /EBC/home/mblesa/ibeat2_test/8018/T2-RO-RS-n3-SkullMask.nii.gz
The file doesn't exist.
Filename = /EBC/home/mblesa/ibeat2_test/8018/T2-RO-RS-n3-SkullMask.nii.gz

ExceptionObject caught!

itk::ImageFileReaderException (0x55dbad6f21d0)
Location: "void itk::ImageFileReader<TOutputImage, ConvertPixelTraits>::GenerateOutputInformation() [with TOutputImage = itk::Image<double, 3>; ConvertPixelTraits = itk::DefaultConvertPixelTraits<double>]"
File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx
Line: 143
Description:  Could not create IO object for reading file /EBC/home/mblesa/ibeat2_test/8018/T2-RO-RS-n3-SkullMask.nii.gz
The file doesn't exist.
Filename = /EBC/home/mblesa/ibeat2_test/8018/T2-RO-RS-n3-SkullMask.nii.gz

Error! File /EBC/home/mblesa/ibeat2_test/8018/T2-RO-RS-n3-SkullMask.nii.gz not exist!

ExceptionObject caught!

itk::ImageFileReaderException (0x559c3492c1d0)
Location: "void itk::ImageFileReader<TOutputImage, ConvertPixelTraits>::GenerateOutputInformation() [with TOutputImage = itk::Image<double, 3>; ConvertPixelTraits = itk::DefaultConvertPixelTraits<double>]"
File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx
Line: 143
Description:  Could not create IO object for reading file /EBC/home/mblesa/ibeat2_test/8018/T2-RO-RS-n3-SkullMask.nii.gz
The file doesn't exist.
Filename = /EBC/home/mblesa/ibeat2_test/8018/T2-RO-RS-n3-SkullMask.nii.gz

ExceptionObject caught!

itk::ImageFileReaderException (0x55adeef3edb0)
Location: "void itk::ImageFileReader<TOutputImage, ConvertPixelTraits>::GenerateOutputInformation() [with TOutputImage = itk::Image<double, 3>; ConvertPixelTraits = itk::DefaultConvertPixelTraits<double>]"
File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx
Line: 143
Description:  Could not create IO object for reading file /EBC/home/mblesa/ibeat2_test/8018/T2-RO-RS-n3-SkullMask.nii.gz
The file doesn't exist.
Filename = /EBC/home/mblesa/ibeat2_test/8018/T2-RO-RS-n3-SkullMask.nii.gz

rm: cannot remove '/EBC/home/mblesa/ibeat2_test/8018/T2-RO-RS-n3-SkullPM.nii.gz': No such file or directory
Skull stripping done!
Doing T1 T2 alignment...
cp: cannot stat '/EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullMask.nii.gz': No such file or directory
cp: cannot stat '/EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped.nii.gz': No such file or directory
Error!!! Input T1 file /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped.nii.gz not exist!
ExceptionObject caught!

itk::ImageFileReaderException (0x55d8a86abdb0)
Location: "void itk::ImageFileReader<TOutputImage, ConvertPixelTraits>::GenerateOutputInformation() [with TOutputImage = itk::Image<double, 3>; ConvertPixelTraits = itk::DefaultConvertPixelTraits<double>]"
File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx
Line: 143
Description:  Could not create IO object for reading file /EBC/home/mblesa/ibeat2_test/8018/brain_mask.nii.gz
The file doesn't exist.
Filename = /EBC/home/mblesa/ibeat2_test/8018/brain_mask.nii.gz

T1 T2 Alignment done!
Doing cerebellum removal...
N3 bias field correction input parameters:  -i '/EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped.nii.gz' -o '/EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-n3.nii.gz' --
Input image: /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped.nii.gz
Output image: /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-n3.nii.gz
ExceptionObject caught!

itk::ImageFileReaderException (0x55e9c08b91d0)
Location: "void itk::ImageFileReader<TOutputImage, ConvertPixelTraits>::GenerateOutputInformation() [with TOutputImage = itk::Image<double, 3>; ConvertPixelTraits = itk::DefaultConvertPixelTraits<double>]"
File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx
Line: 143
Description:  Could not create IO object for reading file /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped.nii.gz
The file doesn't exist.
Filename = /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped.nii.gz

Error! File /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-scaled.nii.gz not exist!

Error! File /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-scaled.nii.gz not exist! -- size: Error! File /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-scaled.nii.gz not exist!,Error! File /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-scaled.nii.gz not exist!,Error! File /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-scaled.nii.gz not exist!, Res: Error! File /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-scaled.nii.gz not exist!,Error! File /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-scaled.nii.gz not exist!,Error! File /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-scaled.nii.gz not exist!

Error open: /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-scaled-forn3.img

Error open: /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-scaled-forn3-n3.img

Error open: /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-scaled-forn3-n3.img

cp: cannot stat '/EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-scaled-forn3.hdr': No such file or directory
Error! File /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-scaled-forn3-n3.hdr not exist!

rm: cannot remove '/EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-scaled.nii.gz': No such file or directory
rm: cannot remove '/EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-scaled-forn3.hdr': No such file or directory
rm: cannot remove '/EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-scaled-forn3.img': No such file or directory
rm: cannot remove '/EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-scaled-forn3-n3.hdr': No such file or directory
rm: cannot remove '/EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-scaled-forn3-n3.img': No such file or directory
ExceptionObject caught!

itk::ImageFileReaderException (0x55725a8e31d0)
Location: "void itk::ImageFileReader<TOutputImage, ConvertPixelTraits>::GenerateOutputInformation() [with TOutputImage = itk::Image<double, 3>; ConvertPixelTraits = itk::DefaultConvertPixelTraits<double>]"
File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx
Line: 143
Description:  Could not create IO object for reading file /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-n3.nii.gz
The file doesn't exist.
Filename = /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-n3.nii.gz

Image Exception : #22 :: ERROR: Could not open image /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-n3-histForCere
terminate called after throwing an instance of 'RBD_COMMON::BaseException'
/bin/edit_image: line 2005:  3365 Aborted                 (core dumped) flirt -in ${strT1FileRORSN3SkullStrippedN3ForCere} -ref ${strT1CereHistTemplate} -out ${strT1FileRORSN3SkullStrippedN3ForCereFlirt} -omat ${strT1FileRORSN3SkullStrippedN3ForCereFlirtMat}
ExceptionObject caught!

itk::ImageFileReaderException (0x5619c44611d0)
Location: "void itk::ImageFileReader<TOutputImage, ConvertPixelTraits>::GenerateOutputInformation() [with TOutputImage = itk::Image<double, 3>; ConvertPixelTraits = itk::DefaultConvertPixelTraits<double>]"
File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx
Line: 143
Description:  Could not create IO object for reading file /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-n3-histForCere-flirt.nii.gz
The file doesn't exist.
Filename = /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-n3-histForCere-flirt.nii.gz

/usr/local/InfantPipeline/bin/convert_mat: line 14: /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-n3-histForCere-flirt.mat: No such file or directory
Cannot read input-matrix
/usr/local/InfantPipeline/bin/CerebellumRemoval_LiModel.py:33: SyntaxWarning: assertion is always true, perhaps remove parentheses?
  assert(dSeg[i]<=dFA[i], 'The output patch size should be smalled than input patch size!')
WARNING: Logging before InitGoogleLogging() is written to STDERR
W0406 09:05:24.138738  3374 _caffe.cpp:122] DEPRECATION WARNING - deprecated use of Python interface
W0406 09:05:24.138787  3374 _caffe.cpp:123] Use this instead (with the named "weights" parameter):
W0406 09:05:24.138808  3374 _caffe.cpp:125] Net('/usr/local/InfantPipeline/bin/LiModule/CerebellumRemoval_T1.prototxt', 1, weights='/usr/local/InfantPipeline/bin/LiModule/CerebellumRemoval_T1.caffemodel')
W0406 09:05:24.145509  3374 upgrade_proto.cpp:71] Note that future Caffe releases will only support input layers and not input fields.
Traceback (most recent call last):
  File "/usr/local/InfantPipeline/bin/CerebellumRemoval_LiModel.py", line 184, in <module>
    main(sys.argv[1:])
  File "/usr/local/InfantPipeline/bin/CerebellumRemoval_LiModel.py", line 151, in main
    imgOrg=sitk.ReadImage(strInImage)
  File "/usr/local/lib/python2.7/dist-packages/SimpleITK/SimpleITK.py", line 8876, in ReadImage
    return _SimpleITK.ReadImage(*args)
RuntimeError: Exception thrown in SimpleITK ReadImage: /tmp/SimpleITK/Code/IO/src/sitkImageReaderBase.cxx:99:
sitk::ERROR: The file "/EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-n3-histForCere-flirt-hist.nii.gz" does not exist.
Image Exception : #22 :: ERROR: Could not open image /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-n3-histForCere
terminate called after throwing an instance of 'RBD_COMMON::BaseException'
/bin/edit_image: line 2005:  3507 Aborted                 (core dumped) flirt -in ${strT1CerebralPMByFlirt} -ref ${strT1FileRORSN3SkullStrippedN3ForCere} -out ${strT1CerebralPM} -applyxfm -init ${strT1FileRORSN3SkullStrippedN3ForCereFlirtMatInverse}
Image Exception : #22 :: ERROR: Could not open image /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-n3-histForCere
terminate called after throwing an instance of 'RBD_COMMON::BaseException'
/bin/edit_image: line 2005:  3508 Aborted                 (core dumped) flirt -in ${strT1CerebellumPMByFlirt} -ref ${strT1FileRORSN3SkullStrippedN3ForCere} -out ${strT1CerebellumPM} -applyxfm -init ${strT1FileRORSN3SkullStrippedN3ForCereFlirtMatInverse}
ExceptionObject caught!

itk::ImageFileReaderException (0x55f8fb1591d0)
Location: "void itk::ImageFileReader<TOutputImage, ConvertPixelTraits>::GenerateOutputInformation() [with TOutputImage = itk::Image<double, 3>; ConvertPixelTraits = itk::DefaultConvertPixelTraits<double>]"
File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx
Line: 143
Description:  Could not create IO object for reading file /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-n3-CerebellumPM.nii.gz
The file doesn't exist.
Filename = /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-n3-CerebellumPM.nii.gz

ExceptionObject caught!

itk::ImageFileReaderException (0x55ce35c7e1d0)
Location: "void itk::ImageFileReader<TOutputImage, ConvertPixelTraits>::GenerateOutputInformation() [with TOutputImage = itk::Image<double, 3>; ConvertPixelTraits = itk::DefaultConvertPixelTraits<double>]"
File: /InfantPipeline/MyProgram/libs/itk/include/ITK-4.13/itkImageFileReader.hxx
Line: 143
Description:  Could not create IO object for reading file /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-n3-CerebellumLabel.nii.gz
The file doesn't exist.
Filename = /EBC/home/mblesa/ibeat2_test/8018/T1-RO-RS-n3-SkullStripped-n3-CerebellumLabel.nii.gz

/bin/edit_image: line 654: /2: syntax error: operand expected (error token is "/2")
Fatal error: Segmentation image cannot be found!

Best regards,

Manuel

[mblesac]

Hi @Zhengwang-Wu,

Did you have any progress on this?

On the other hand I maybe able to update my GPU, which one you would recommend? Thanks in advance.

Best regards,

Manuel

[Zhengwang-Wu]

Hi, @mblesac, we plan to release a version to support older GPUs for maximal capability at the end of this week. I think either way is ok, depending on your schedule.

[mblesac]

Hi, That's great! Thanks! I'll test it then.

[Zhengwang-Wu]

Hi, @mblesac, I uploaded a version with tag: ibeat_v2:release115. Since I do not have a older GPU, can you have a try to see whether this version works for you. Just get it by: docker pull ibeatgroup/ibeat_v2:release115.

[mblesac]

Hi,

Thanks for this! It works! I tested it for one subject and the results look great! See a couple of images:

Do you know when the next steps of cortical and cerebellum parcellation will be added? Thanks in advance.

Best regards,

Manuel

[sam-osia]

Hi @Zhengwang-Wu,

I'm having similar issues with release 2.0.5. I'm fairly new, so apologies if I don't provide enough detail:

ubuntu@cranio-gpu-1:~$ docker images
REPOSITORY            TAG                       IMAGE ID       CREATED         SIZE
ibeatgroup/ibeat_v2   release205                21ccd37f96df   2 months ago    15.5GB
nvidia/cuda           11.2.2-base-ubuntu18.04   6e5cbaa46a88   20 months ago   113MB
ubuntu@cranio-gpu-1:~$ docker run --rm --gpus=all nvidia/cuda:11.2.2-base-ubuntu18.04 nvidia-smi
Fri Jul 12 13:36:44 2024       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.108.03   Driver Version: 510.108.03   CUDA Version: 11.6     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla V100-PCIE...  On   | 00000000:00:05.0 Off |                    0 |
| N/A   28C    P0    25W / 250W |      0MiB / 32768MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+
ubuntu@cranio-gpu-1:~$ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Sun_Feb_14_21:12:58_PST_2021
Cuda compilation tools, release 11.2, V11.2.152
Build cuda_11.2.r11.2/compiler.29618528_0
ubuntu@cranio-gpu-1:~$ sudo docker run --gpus=all --rm -it -v /hpf/projects/ccmuhn/sam/brain_age/tmp/ibeat_test:/InfantData --user $(id -u):$(id -g) ibeatgroup/ibeat_v2:release205 --t1 bc0043_20010101_152817_4_mri_3DT1_V01.nii --age 1
************************************************
**iBEAT V2.0 pipeline processing**
************************************************
Pipeline input parameters:  --t1 'bc0043_20010101_152817_4_mri_3DT1_V01.nii' --age '1' --
input age:1
Warning: No output folder desinated. Will create a result folder under current directory to save outputs!
Warning: Subject name cannot be determined. Use image file name instead!
T1w image: bc0043_20010101_152817_4_mri_3DT1_V01.nii
No T2w image inputed!
Model age in months: 1
Output folder: /InfantData/Result/bc0043_20010101_152817_4_mri_3DT1_V01
Orientation code: T1w-RPS
Doing reorientation and resampling...
Original T1 resolution in RAI: 1.041667x1.000000x1.041667. New T1 resolution: 0.8x0.8x0.8
Original T1 size in RAI: 96x120x192. New T1 size: 125x150x250
Reorientation and resampling done!
Doing N3 bias field correction...
N3 bias field correction done!
Doing skull striping...
/usr/local/InfantPipeline/bin/Skullstripping_LiModel.py:38: SyntaxWarning: assertion is always true, perhaps remove parentheses?
  assert(dSeg[i]<=dFA[i], 'The output patch size should be smalled than input patch size!')
WARNING: Logging before InitGoogleLogging() is written to STDERR
W0712 13:37:08.934562   101 _caffe.cpp:122] DEPRECATION WARNING - deprecated use of Python interface
W0712 13:37:08.934613   101 _caffe.cpp:123] Use this instead (with the named "weights" parameter):
W0712 13:37:08.934618   101 _caffe.cpp:125] Net('/usr/local/InfantPipeline/bin/LiModule/Skullstripping_T1_downsampled.prototxt', 1, weights='/usr/local/InfantPipeline/bin/LiModule/Skullstripping_T1_downsampled.caffemodel')
W0712 13:37:08.937228   101 upgrade_proto.cpp:71] Note that future Caffe releases will only support input layers and not input fields.
In image: /InfantData/Result/bc0043_20010101_152817_4_mri_3DT1_V01/T1-rescaled-RO-RS-n4-odd-downsample-hist-z200.nii.gz 
Out image: /InfantData/Result/bc0043_20010101_152817_4_mri_3DT1_V01/T1-rescaled-RO-RS-n4-odd-downsample-hist-z200-skullpm.nii.gz 
Model path: /usr/local/InfantPipeline/bin/LiModule/Skullstripping_T1_downsampled.caffemodel
Network structure path: /usr/local/InfantPipeline/bin/LiModule/Skullstripping_T1_downsampled.prototxt
('Input patch size:  ', [64, 64, 64])
('Output patch size: ', [64, 64, 64])
('Step size: ', [12, 12, 12])
**F0712 13:37:13.552186   101 im2col.cu:497] Check failed: error == cudaSuccess (209 vs. 0)  no kernel image is available for execution on the device**
*** Check failure stack trace: ***
/bin/inc_func_operation: line 169:   101 Aborted                 (core dumped) CUDA_VISIBLE_DEVICES=${gpu_id} python ${pipeline_root}/Skullstripping_LiModel.py -i ${t1_file_rors_n4_odd_downsample_hist_z200} -m ${skullstripping_model} -p ${skullstripping_proto} -o ${t1_file_rors_n4_odd_downsample_hist_z200_skullpm} --size_in ${patch_size_in} --size_out ${patch_size_out} --size_step ${step_size}

Thank you in advnace!

[trichtu]

Hi, I have the same cuda issue with version 2.05.

My cuda version in system is shown below:

[trichtu]

Also, could you develop a version that without using the GPU. So there would be no CUDA issues at all.