search

bkainz / fetalReconstruction

GPU accelerated source code for motion compensation of multi-stack MRI data
57 stars 28 forks source link

PVRresonstructionGPU runtime error: "CUDA Error in patchBasedPSFReconstructionKernel(), line 132: too many resources requested for launch" #19

Open dittothat opened 5 years ago

dittothat commented 5 years ago

Trying to get a Docker image set up so this code is more portable. We are making progress, but stuck on a runtime error in PVRreconstructionGPU. Has anyone encountered a similar issue? Any thoughts?

PVRreconstructionGPU --output PVR1.nii --input 10_3T_nody_001.nii 10_3T_nody_002.nii 14_3T_nody_001.nii 14_3T_nody_002.nii 21_3T_nody_001.nii 21_3T_nody_002.nii 23_3T_nody_001.nii 23_3T_nody_002.nii --resolution 1 --patchSize 32 32 --patchStride 16 16 CUDA Error in patchBasedPSFReconstructionKernel(), line 132: too many resources requested for launch

A friend with C++ know how helped me a bit: After several combinations of testing, I think the author forgot to allocate memory and resulted in "too many resources requested for launch" error.

  1. I tested on the github-provided data with PVRreconstructionGPU --output PVR1.nii --input 10_3T_nody_001.nii 10_3T_nody_002.nii 14_3T_nody_001.nii 14_3T_nody_002.nii 21_3T_nody_001.nii 21_3T_nody_002.nii 23_3T_nody_001.nii 23_3T_nody_002.nii --resolution 1 --patchSize 32 32 --patchStride 16 16 and got the same error.

  2. I tested on the github-provided data with no overlapping data:

    PVRreconstructionGPU --output PVR1.nii --input 10_3T_nody_001.nii 10_3T_nody_002.nii 14_3T_nody_001.nii 14_3T_nody_002.nii 21_3T_nody_001.nii 21_3T_nody_002.nii 23_3T_nody_001.nii 23_3T_nody_002.nii --resolution 1 --patchSize 32 32 --patchStride 32 32 and got the same error.

  3. I commented out "if(useSpx)-else block" in initPatchBasedRecon_gpu.patchBasedPatchInitKernel (line 71-85 in initPatchBasedRecon_gpu.cu) on our-data and can finish the program but complaint with no initialization.

  4. I dug into the content in "if(useSpx)-else block", it is "reconstruction.isMasked" and "inputStack.setPatchValue" which are the functions to check and set some value at some specific location.

bkainz commented 5 years ago

thanks a lot for your effort! This part also needs refactoring. I'll try to revisit this with a student next term. Currently, we are more focused on a one-stop-shop solution for the reconstruction pipeline. I'll post the outcome of this here as soon as available.

dittothat commented 5 years ago

Do you have any sense of how involved this fix would be? Our institution has a lot of interest in the PVR technique. I am considering trying to work on this issue. As I have experimented with other compute architecture compile settings, this error message has become slightly more specific, so I think I might be able to work on it. My concern based on your "needs refactoring" comment is that this error is only the tip of the iceberg.

bkainz commented 5 years ago

yes, you're right. I haven't worked with this code since years, only with the binaries on a fozen production machine, so I cannot gurantee that there aren't more issues. I don't know your timeline but I plan to find a final year student next term who would be interested to work on this, also integrating modern ML techniques. However, this will likely need 6 months.

dittothat commented 5 years ago

Thanks for the information. I will let you know if I make progress.