Open OlivierBakker opened 3 months ago
bnorthan
commented
3 months ago How big is your image? My first thought is that you might be running out of memory (the non-circulant version uses more memory). If you are running out of memory you could try the dask version. Here is an example as to how to use the dask version https://github.com/clij/clij2-fft/blob/master/python/tests/test_richardson_lucy_dask.py.
If not a memory problem let me and I'll look into it further.
OlivierBakker
commented
3 months ago Hi,
Thanks very much for the quick response! Not so sure it is a memory issue, as I the test image is very small (14,500,500), psf (31,47,47) and the gpu's we run have 80G of memory. I also tried it with a small array of np.ones of dimension (100, 100, 24) with psf (24, 24, 12) with the same result.
See log below:
Python 3.10.0 | packaged by conda-forge | (default, Nov 20 2021, 02:24:10) [GCC 9.4.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.12.3 -- An enhanced Interactive Python. Type '?' for help.
In [1]: import tifffile
...: import pyopencl as cl
...: from clij2fft.richardson_lucy import richardson_lucy, richardson_lucy_nc
...: import numpy as np
...:
...: platforms = cl.get_platforms()
...:
...: devices=platforms[0].get_devices()
...:
...: for device in devices:
...: print(device)
...: print(device.get_info(cl.device_info.GLOBAL_MEM_SIZE))
...:
...:
...: img = tifffile.imread("img.tiff")
...: psf = tifffile.imread("psf.tiff")
...:
...: img.shape
...: psf.shape
<pyopencl.Device 'NVIDIA A100-SXM4-80GB' on 'NVIDIA CUDA' at 0x557a56c3f590>
84974239744
Out[1]: (31, 47, 47)
In [2]: img.shape
Out[2]: (14, 500, 500)
In [3]: psf.shape
Out[3]: (31, 47, 47)
In [4]: res = richardson_lucy(img, psf, 10, 0)
get lib
platform 0 NVIDIA CUDA
device name 0 NVIDIA A100-SXM4-80GB
Richardson Lucy Started
0
Richardson Lucy Finished
In [5]: res = richardson_lucy_nc(img, psf, 10, 0)
get lib
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
Cell In[5], line 1
----> 1 res = richardson_lucy_nc(img, psf, 10, 0)
File lib/python3.10/site-packages/clij2fft/richardson_lucy.py:128, in richardson_lucy_nc(img, psf, numiterations, regularizationfactor, lib, platform, device)
125 lib = getlib()
127 # the normalization factor is the valid region correlated with the PSF
--> 128 lib.convcorr3d_32f(int(normal.shape[2]), int(normal.shape[1]), int(normal.shape[0]), valid, shifted_psf, normal,1,platform,device)
130 # get rid of any zeros in the normal to avoid divide by zero issues
131 #normal[normal<0.00001]=1
132
133 # deconvolution using clij2fft
134 if regularizationfactor==0:
RuntimeError: ffi_prep_cif_var failed
In [6]: img = np.ones((100, 100, 24), dtype=np.float32)
In [7]: psf = np.ones((24, 24, 12), dtype=np.float32)
In [8]: res = richardson_lucy(img, psf, 10, 0)
get lib
platform 0 NVIDIA CUDA
device name 0 NVIDIA A100-SXM4-80GB
2 warnings generated.
2 warnings generated.
2 warnings generated.
Richardson Lucy Started
0
Richardson Lucy Finished
In [9]: res = richardson_lucy_nc(img, psf, 10, 0)
get lib
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
Cell In[9], line 1
----> 1 res = richardson_lucy_nc(img, psf, 10, 0)
File lib/python3.10/site-packages/clij2fft/richardson_lucy.py:128, in richardson_lucy_nc(img, psf, numiterations, regularizationfactor, lib, platform, device)
125 lib = getlib()
127 # the normalization factor is the valid region correlated with the PSF
--> 128 lib.convcorr3d_32f(int(normal.shape[2]), int(normal.shape[1]), int(normal.shape[0]), valid, shifted_psf, normal,1,platform,device)
130 # get rid of any zeros in the normal to avoid divide by zero issues
131 #normal[normal<0.00001]=1
132
133 # deconvolution using clij2fft
134 if regularizationfactor==0:
RuntimeError: ffi_prep_cif_var failedThanks for the additional feedback. I changed the interface to convcorr3d a few months ago and somehow did not update the Linux libs. I think I've fixed it, but let me know. Update clij2-fft, confirm that you have version 0.27 and try again.
Once you get it going I'd be interested to hear if the non-circulant version makes a difference for your image. I noticed your PSF size in z is longer than the image size. Is that correct? The non-circulant version should be able to handle that, however I haven't done a lot of testing with that scenario. What type of instrument are you using and how did you create the PSF?
OlivierBakker
commented
2 months ago Hi,
Thanks for your very quick fix, it solves the issue, but I am now getting another error using v0.27 when calling the nc version:
Runtime error: ret returned -9999 at /home/bnorthan/code/i2k/clij/clij2-fft/native/clij2fft/clij2fft.cpp:1269cIs this some kind of broken reference? I installed v0.27 through pip.
The image I get out seems to have a constant value for all pixels. The circulant version seems to work still. I also get this issue when I use the array of 1's.
Some off topic context:
I noticed your PSF size in z is longer than the image size. Is that correct?
With regards to the PSF, that is correct as I have not trimmed our psf here (There are some completely black planes that I should probably trim), the majority of the signal is within +-4 planes of the center of mass (more detaills below).
Some context of what we are hoping to do that are maybe a bit of topic (and maybe better moved to image sc?), please don't feel obligated to go through this, but as I am still learning about deconvolution, some expert advice would be welcome!
We are in a situtation where we run image aquistiaiton in high throughput on small cells (5 to 15um) which are not in a clean monolayer, so are a little limeted in how many z-planes we can aquire while keeping things manageble computationally (every plane can add about 500gb). We are planning to up the number of z-planes a bit in future experiments, but we will likely always have some edges in the z axis. Having experimented a bit however, in this non ideal senario where we have 0.5 / 0.6 plane spacing with 14-16 planes and some strong edges as we started the aquistion 0.5um up, the standard RL seems to work suprisingly well with suprsingly minimal artefacts, at least using the clij2 implementation (Below an example of the nuclei before and after). I didn't have much luck with redlionfish, as that gave strong artefacts, even with padding zeroes to the image.
However if you feel in this senerio where we will have cells with strong edges, attempting decon is not sensible, please let us know! If it would be better to capture more of the cell by increasing our z-spacing to lets say 0.6 or 0.7 (slightly over the Nyquist threshold of 0.5) but then we capture more of the cells, and have fewer edges, that would also be helpfull.
What type of instrument are you using and how did you create the PSF?
We are running an Opera Phenix, PSFs were generated from beads using averaging (more detaills below). I found that trimming the PSF too agressively seems to introduce some artefacts, and the results don't look as good as when I just included the whole thing. I have not tested yet just trimming the empty areas.
With the gamma at 0.2 to illustrate where the non-zero signal is:
With the gamma at 1:
Before:
After 100 iterations:
Detaills PSF generation: Bead images were generated on z spacing of 0.1um, then selecting single beads without close neighbours, normalizing indiviual bead images, trimming outliers in intensity and then registering and averaging beads. Finally I center arround the center of mass and substract the background signal to make sure that is set at zero and normalize so the max intensity is one. As the image here is at 0.6um I sample every 6 planes starting from the center of the PSF image as the final PSF.
bnorthan
commented
2 months ago Hi @OlivierBakker
Your images from circulant RL actually pretty good, so not sure the non-circulant version will help you much. That being said I'd still like to figure out why it isn't working for you.
If you are able to post a sample of an image and one of your PSFs I can troubleshoot further. Either here or on ImageSC.
imagesc-bot
commented
2 months ago This issue has been mentioned on Image.sc Forum. There might be relevant details there:
https://forum.image.sc/t/issues-with-python-version-of-clij2-richardson-lucy-nc/101300/1
Hi,
I am running into a runtime error during calling the non circular RL from python version, where it throws RuntimeError: ffi_prep_cif_var failed
The circular version richardson_lucy() does seem to work fine and produces quite good results. Any advice on if this could be a data issue (psf / data in wrong format) or a install issue would be appreciated as I'd like to test the nc version as I have some strong edges in my data.
I tried a bit of googling and looking at the source, and it seems like its an error between the python - C interaction but it goes a bit above my head.
Running on Ubuntu 22.04.4 with version python 3.10.0, clij2-fft 0.26, pyopencl 2024.2.7 and cuda 12.1 (full package list below).
EDIT: I have tried running with np.ones to rule out an issue with my image / psfs following from: https://github.com/clij/clij2-fft/blob/master/python/interactive%20tests/test_richardson_lucy.py
It still gives the same issue
Package versions in my environment: