SPECT reconstruction on GPU

[niinimaki]

Hi,

I discovered that the SPECT reconstruction on GPU fails for any volume other than 128 x 128 x 128. On CPU, the SPECT reconstruction seems to work for different volumes. Looking more into it, it seems that the forward projection (with or without GPU) works as expected for all volumes but the backward projection seem to work consistently (for different sizes) only on CPU.

Do you have any ideas on this issue?

[moralesq]

I noticed that that the NiftyRec.py function SPECT_backproject_parallelholes() does something funky with the sizes. Specifically, it defines the backprojection (i.e., image) size as: int32( [projection.shape[0],projection.shape[1],projection.shape[0]] )

Why is the third dimension projection.shape[0]] ?

[txq0530]

I also have the same problem. How did you solve this? I have two 12GB GPU in the computer, but both of them did not work.

[mscipio]

Are you sure you are using the latest version of the NiftyRec library supported by occiput? I would suggest all of you to try to compile NiftyRec 3.0.0 from here and check again. There is not full compatibility with the previous versions of NiftyRec, at least for the PET part ... I am sorry but I am not really familiar with the SPECT recon module.

Also posting the exact error you get (usually inside the terminal/log file associated with jupyter notebook server you are using) could be of some help.

[txq0530]

Many thanks for your explanation and suggestion. For my case it works perfect on CPU with different image sizes. The problem is that on GPU it only works with size of 128x128x128.

As suggested I installed both NiftyRec 3.0.0 and occiput-1.0.12-py2.7 from source, on Ubuntu 16.04 with Cuda 8.0 for two 12Gb TITAN X (Pascal).

When I first tried the phantom example in ../notebook/ with image size >=256x256x256, it gave the error: [NiftyReg CUDA ERROR] file "...../emission-lib_gpu/_et_accumulate_gpu.cu" in line 26 : invalid argument. For image size < 256x256x256, only 128x128x128 works. Then for other sizes which are smaller than 256x256x256, the warnings came in between the iteration steps (set as 10 iteration) as: "....../SPECT.py: 418: RuntimeWarning: overflow encountered in multiply activity = activity update # self.get_mask().data". In this case the maximum and minimum value of the reconstruction result is separately Inf and 0. This should not be the case for a correct phantom reconstruction.

[txq0530]

Dear author, We do appreciate your work on the NiftyRec iterative reconstruction. Could you please help us find a solution about our problem?

We are now running the SPET phantom example in .../occiput/notebooks/Example_SPECT_reconstruction.py. The package versions are: NiftyRec 3.0.0, NiftyPy 0.4, occiput-1.0.12-py2.7.

To further explore the reason why only 128x128x128 sized image can be run correctly on GUP, we set the phantom image size as 64x64x64 for comparison. For forward projection, both GPU and CPU produced the same results (checked the Max/Min/voxel(30,30,30) ) for both image sizes. For back projection, GPU and CPU implement produce the same results only for image of 128x128x128. While for image of size 64x64x64, single implement of "backproject()" function (within function ''SPEC.estimate_activity()") on the same projection data produced different backprojection data(checked in the same way as above for forward projection). Going deep into the package, I found it came from the different results of function "SPECT_backproject_paralleholes()" on GPU and CPU when the image size is not 128x128x128.

Many thanks if you can help to find a way to solve this. If you are not managing SPECT, could you please recommend me the right author who are familiar with it? Thanks again and look forward to your response.

[mscipio]

I am sorry, but as I said before, I have never really worked (or even tested) the SPECT module, and I don't really have time to do it now.

If you look at the PET part of occiput you will see that there have been many changes in the latest releases (i.e. we don't use anymore the method estimate_activity() for the reconstruction, which has been replaced by more meaningful names like osem_reconstruction or mlem_reconstruction): it can be that some of the changes made and tested on the PET part have not been mirrored to the SPECT module, so something now is broken over there.

Even the fact that CPU processing is still working for you while not being officially supported anymore for quite some time is quite surprising to me.

Only thing I can do is to tag @spedemon here, and we will see if he has some spare time to follow up on your issue. If I happen to find time to do some testing myself, I will let you know. But I am currently in the final stages of my Ph.D., and I hope you will understand I won't have much time to play a lot with things not directly related to my thesis work ... sorry about this!

[txq0530]

Thanks a lot. I will try the PET part.