search

clij / clij2-fft

21 stars 6 forks source link

Function 'cleanup' not found when running test_richardson_lucy_dask.py with the live version #33

Open boydcpeters opened 4 months ago

boydcpeters commented 4 months ago

Hello!

I am trying to get the live version of clij2-fft to work in Python on Windows. I have cloned the repository clij2-fft repository and have edited the paths in the copylibsconda.sh file so it copies the libraries to the correct anaconda environment. Afterwards, I ran the editableinstall.sh file. I am happy to confirm that the package was installed and that all the Windows libraries are correctly copied into my Anaconda library. However, when I try to run the interactive test: test_richardson_lucy_dask.py I get the following error:

Traceback (most recent call last):
  File "c:\Users\boydc\Projects\university\mep\mep_data_analysis\test_richardson_lucy_dask.py", line 49, in <module>
    decon = richardson_lucy_dask(
            ^^^^^^^^^^^^^^^^^^^^^
  File "c:\users\boydc\projects\packages\clij2-fft\python\clij2fft\richardson_lucy_dask.py", line 205, in richardson_lucy_dask
    lib.cleanup()
    ^^^^^^^^^^^
  File "C:\Users\boydc\anaconda3\envs\clij2-env\Lib\ctypes\__init__.py", line 389, in __getattr__
    func = self.__getitem__(name)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\boydc\anaconda3\envs\clij2-env\Lib\ctypes\__init__.py", line 394, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: function 'cleanup' not found

However, the deconvolution does seem to run as can be seen with the printed output if debug=True in the richardson_lucy_dask() function:

image shape is (356, 356, 228)
Platform: NVIDIA CUDA has 1 devices
     <pyopencl.Device 'Quadro P1000' on 'NVIDIA CUDA' at 0x1d7c776c1b0>
Platform: Intel(R) OpenCL HD Graphics has 1 devices
     <pyopencl.Device 'Intel(R) UHD Graphics 630' on 'Intel(R) OpenCL HD Graphics' at 0x1d7c7f96260>
richardson_lucy_dask

image size (356, 356, 228)
psf size (128, 128, 64)
gpu mem is  3.99981689453125
rl mem is  1.6348734498023987
chunk factor is  2.0
chunk size is (356, 178.0, 114.0)

start rlnc
gpu num is 0
block id is None
block info is None
Traceback (most recent call last):
  File "c:\users\boydc\projects\packages\clij2-fft\python\clij2fft\richardson_lucy_dask.py", line 180, in rl_dask_task
    result=richardson_lucy_nc(img, psf, numiterations, regularizationfactor=regularizationfactor, lib=lib, platform = platform, device = device_num)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\users\boydc\projects\packages\clij2-fft\python\clij2fft\richardson_lucy.py", line 140, in richardson_lucy_nc
    extended_size = get_next_smooth(extended_size)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\users\boydc\projects\packages\clij2-fft\python\clij2fft\pad.py", line 62, in get_next_smooth
    return tuple(map(lambda i: next_smooth(i), size))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\users\boydc\projects\packages\clij2-fft\python\clij2fft\pad.py", line 62, in <lambda>
    return tuple(map(lambda i: next_smooth(i), size))
                               ^^^^^^^^^^^^^^
  File "c:\users\boydc\projects\packages\clij2-fft\python\clij2fft\pad.py", line 36, in next_smooth
    z = int(10*math.log2(x))
               ^^^^^^^^^^^^
ValueError: math domain error

EXCEPTION math domain error
putting gpu num back 0

start rlnc
gpu num is 0
block id is []
block info is []
Traceback (most recent call last):
  File "c:\users\boydc\projects\packages\clij2-fft\python\clij2fft\richardson_lucy_dask.py", line 180, in rl_dask_task
    result=richardson_lucy_nc(img, psf, numiterations, regularizationfactor=regularizationfactor, lib=lib, platform = platform, device = device_num)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\users\boydc\projects\packages\clij2-fft\python\clij2fft\richardson_lucy.py", line 140, in richardson_lucy_nc
    extended_size = get_next_smooth(extended_size)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\users\boydc\projects\packages\clij2-fft\python\clij2fft\pad.py", line 62, in get_next_smooth
    return tuple(map(lambda i: next_smooth(i), size))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\users\boydc\projects\packages\clij2-fft\python\clij2fft\pad.py", line 62, in <lambda>
    return tuple(map(lambda i: next_smooth(i), size))
                               ^^^^^^^^^^^^^^
  File "c:\users\boydc\projects\packages\clij2-fft\python\clij2fft\pad.py", line 36, in next_smooth
    z = int(10*math.log2(x))
               ^^^^^^^^^^^^
ValueError: math domain error

EXCEPTION math domain error
putting gpu num back 0

start rlnc
gpu num is 0
block id is (0, 1, 1)
block info is {0: {'shape': (356, 376, 248), 'num-chunks': (1, 2, 2), 'array-location': [(0, 356), (188, 376), (124, 248)], 'chunk-location': (0, 1, 1)}, None: {'shape': (356, 376, 248), 'num-chunks': (1, 2, 2), 'array-location': [(0, 356), (188, 376), (124, 248)], 'chunk-location': (0, 1, 1), 'chunk-shape': (356, 188, 124), 'dtype': <class 'numpy.float32'>}}
calling convcorr 0 0

platform 0 NVIDIA CUDA
device name 0 Quadro P1000

Richardson Lucy Started
0 10 20 30 40 
Richardson Lucy Finished
putting gpu num back 0

start rlnc
gpu num is 0
block id is (0, 1, 0)
block info is {0: {'shape': (356, 376, 248), 'num-chunks': (1, 2, 2), 'array-location': [(0, 356), (188, 376), (0, 124)], 'chunk-location': (0, 1, 0)}, None: {'shape': (356, 376, 248), 'num-chunks': (1, 2, 2), 'array-location': [(0, 356), (188, 376), (0, 124)], 'chunk-location': (0, 1, 0), 'chunk-shape': (356, 188, 124), 'dtype': <class 'numpy.float32'>}}
calling convcorr 0 0

platform 0 NVIDIA CUDA
device name 0 Quadro P1000

Richardson Lucy Started
0 10 20 30 40 
Richardson Lucy Finished
putting gpu num back 0

start rlnc
gpu num is 0
block id is (0, 0, 1)
block info is {0: {'shape': (356, 376, 248), 'num-chunks': (1, 2, 2), 'array-location': [(0, 356), (0, 188), (124, 248)], 'chunk-location': (0, 0, 1)}, None: {'shape': (356, 376, 248), 'num-chunks': (1, 2, 2), 'array-location': [(0, 356), (0, 188), (124, 248)], 'chunk-location': (0, 0, 1), 'chunk-shape': (356, 188, 124), 'dtype': <class 'numpy.float32'>}}
calling convcorr 0 0

platform 0 NVIDIA CUDA
device name 0 Quadro P1000

Richardson Lucy Started
0 10 20 30 40 
Richardson Lucy Finished
putting gpu num back 0

start rlnc
gpu num is 0
block id is (0, 0, 0)
block info is {0: {'shape': (356, 376, 248), 'num-chunks': (1, 2, 2), 'array-location': [(0, 356), (0, 188), (0, 124)], 'chunk-location': (0, 0, 0)}, None: {'shape': (356, 376, 248), 'num-chunks': (1, 2, 2), 'array-location': [(0, 356), (0, 188), (0, 124)], 'chunk-location': (0, 0, 0), 'chunk-shape': (356, 188, 124), 'dtype': <class 'numpy.float32'>}}
calling convcorr 0 0

platform 0 NVIDIA CUDA
device name 0 Quadro P1000

Richardson Lucy Started
0 10 20 30 40 
Richardson Lucy Finished
putting gpu num back 0
Execution time of rl dask multi gpu: 41.39972186088562 seconds
Traceback (most recent call last):
  File "c:\Users\boydc\Projects\university\mep\mep_data_analysis\test_richardson_lucy_dask.py", line 49, in <module>
    decon = richardson_lucy_dask(
            ^^^^^^^^^^^^^^^^^^^^^
  File "c:\users\boydc\projects\packages\clij2-fft\python\clij2fft\richardson_lucy_dask.py", line 205, in richardson_lucy_dask
    lib.cleanup()
    ^^^^^^^^^^^
  File "C:\Users\boydc\anaconda3\envs\clij2-env\Lib\ctypes\__init__.py", line 389, in __getattr__
    func = self.__getitem__(name)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\boydc\anaconda3\envs\clij2-env\Lib\ctypes\__init__.py", line 394, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: function 'cleanup' not found

Do you have any idea what could be going wrong?

I have also installed clij2-fft via PyPI in a different conda environment and when I run the test_richardson_lucy_dask.py file with some of the kwargs removed as they are not yet in the package on PyPi then the file does run. So I am able to get the dask version to work, but the live version has multi-GPU support and this is something I would like to try out.

bnorthan commented 4 months ago

It could be because the compiled libraries on github have not been updated.

This isn't documented well, but if you run the hackinstall script it calls both the cpp build (cppbuld.sh) and the edibableinstall.sh. So if it works you should have the newest version of the native libary.

There is some description of how to set up the build here.

Let me know if you run into any issues. I haven't documented everything as well as I would have liked, but I am usually happy to give support here or on the Image.sc forum.

boydcpeters commented 4 months ago

Thanks for the quick reply! Once I have some time again, I will try it out and if I run into issues I will let you know.

boydcpeters commented 3 days ago

Hi, apologies for the delayed reply. Unfortenately, our data turned out to be too noisy to carry out any meaningful deconvolution, so I set this project aside while finishing my master's thesis.

That said, I appreciated your help, so I wanted to revisit it. After running hackinstall.sh (which appears to perform a Python build followed by an editable install), I still encountered the same error as before.

When I print out lib and dir(lib) inside the richardson_lucy_dask function, I get the following:

Library loaded: <CDLL 'clij2fft.dll', handle 7fffca3c0000 at 0x1e153d0a510>  
Available methods in library: ['_FuncPtr', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattr__', '__getattribute__', '__getitem__', '__getstate__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_func_flags_', '_func_restype_', '_handle', '_name', 'convcorr3d_32f', 'deconv3d_32f', 'deconv3d_32f_tv']

I hope this information is helpful if you’re continuing to work on it. And would you like me to close the issue or keep it open?