search

fiji / SPIM_Registration

Deprecated, instead check out: https://github.com/preibischLab/multiview-reconstruction & https://github.com/PreibischLab/BigStitcher
GNU General Public License v2.0
23 stars 22 forks source link

Fiji exits when deconvolving entire image at once with CUDA #116

Closed lacan closed 7 years ago

lacan commented 7 years ago

Hi,

the title has most of the information, On every machine we tried, deconvolving the entire image at once crashes the entire java application.

this crash happens after all views are loaded, transformed and ready for deconvolution.

It does not seem to be a memory issue as when running out of memory on the GPU gives me an error message on the console about an invalid memory address.

I am deconvolving a ~ 900 x 1000 x 1000 px image on an 8GB graphics card, but this crash happens with smaller datasets... I have to do the deconvolution by blocks.

Has anyone experienced something similar and what steps could I undertake to try and fix this?

StephanPreibisch commented 7 years ago

Yes, CUDA does not catch when it runs out of memory. So quite sure this is a memory issue on your GPU. The cuFFT implementation is not particularly memory efficient, so especially if the block sizes are not power of 2 (which is the case if you do the whole image at once), it uses a lot. You can try to do one block which is bigger than the actual bounding box, this might work.

So for example you bounding box is 891x699x480, try using blocks, block size 1024x768x512 or even 1024x1024x512.

lacan commented 7 years ago

Hi, thank for the reply, I will gladly try this today and get back to you!

lacan commented 7 years ago

I've managed to reduce it to about 20 blocks, using about 4GB of RAM. It seems that iof the block is larger, it does take up more than 8GB for this dataset. Thank you again