search

clij / clij2

GPU-accelerated image processing for everyone
https://clij.github.io/clij2
Other
48 stars 14 forks source link

CL_MEM_OBJECT_ALLOCATION_FAILURE on laptop with integrated AND dedicated graphics #28

Closed krizzodil closed 3 years ago

krizzodil commented 3 years ago

clij_problem_exception.txt clij_problem_info.txt clij_problem.csv.txt

clij_problem

Hello!

I'm experiencing an issue with, i guess, GPU RAM usage. I tried to run a clij2 gaussian filter, radius x30y30 on a rather large stack of approx 2359x2363x163 16 bit, 1.7GB, but got an OpenCL error: -4 -> CL_MEM_OBJECT_ALLOCATION_FAILURE

It might be because I'm running a gaming laptop with, both, an integrated GPU, and a dedicated GPU. Maybe clij gets confused by the readouts. It's an Asus TUF FX705 with an AMD Ryzen 7 3750H and an NVidia GeForce GTX 1660 Ti 6 GB GDDR6.

I tried to include all the information in the screenshot, but forgot the clij versions. I installed it today via FIJI update sites: clij 1.7.5.1 clij2 2.2.0.11 clijx 0.30.1.0 ImageJ 1.53c

I can easily reproduce the error, in case you need more information.

Thank you! Christian

p.s.: CLIJ is crazy! I'm also looking forward to clesperanto, especially for Python! I'm still trying to make the prototype work on my computer (edit: It works now! It's nice :) ).

edit: Now with text-files

haesleinhuepf commented 3 years ago

Hi @krizzodil ,

I'm glad to hear you enjoy using clij. :-)

When working with large images and large kernel on Windows, it's possible that the operating system kills operations before they finish. Windows is afraid that the GPU is in an endless loop and stops execution. In order to increase the timeout, please follow the instructions on this site: https://clij.github.io/clij2-docs/troubleshooting

Let me know if this solves the issue!

Best, Robert

krizzodil commented 3 years ago

clij_problem_2

Hello @haesleinhuepf !

I've now followed the first entry, "Big Images on NVidia graphics cards", and tried again, after restarting my computer. The problem persists. I also tried a higher value (ac), with the same outcome. Do expect that the clijx RAM monitor displays over-occupancy? I'm actually not sure if I am reporting an issue, or something expected with large images.

In Python I get a similar (same?) error using cle.gaussian_blur. In short: File "", line 90, in enqueue_knl_gaussian_blur_separable_3d pyopencl._cl.MemoryError: clEnqueueNDRangeKernel failed: MEM_OBJECT_ALLOCATION_FAILURE I just solved it by separating the stack into chunks using numpy.split() and processing them one after the other. Of course, this would also be an option in FIJI, though a bit awkward.

Best wishes, Christian

haesleinhuepf commented 3 years ago

Hi Christian @krizzodil ,

I think I can see the issue. Please execute the clinfo macro to determine the capabilities of your GPU:

image

You can read in the line "MaxMemoryAllocationSizeInBytes" how large an image can be. This value depends on the vendor / model / driver and is typically a quarter of the GPU memory. Thus, your image is something like 100 MB too large. Consider cropping the image. Alternatively, working with an 8-bit image of the same dimensions might be doable.

Can you confirm that it works if your crop the image down to 1.5 GB?

Best, Robert

krizzodil commented 3 years ago

cl_info.txt gauss2D

I wrote a script to test different NSlices. It only started to work once it was down to 0.7 GB. You can find the script and output in the screenshot. The radius of the gauss didn't change the success, only the processing time. Best wishes, Christian

haesleinhuepf commented 3 years ago

Glad that you figured out how to make it work. Just two minor comments:

May I close this issue? 🙃

krizzodil commented 3 years ago

Ah, I understand! Builtin splitting of input-data would not be a favor to the user, as this would cost a lot of performance decrease when processing multiple steps. (Though, in case of the GUI it might be interesting. Yet, in turn, it would create a discrepancy between GUI-calls and scripting-calls). Unfortunately I missed the opportunity to register for your I2K workshop, and many others. I'm still hoping for recordings ;)

Yes, the issue, especially in my head, is closed.

Liebe Grüße, & schöne Feiertage, Christian

krizzodil commented 3 years ago

Oh, and thank you so much for your time! And my apologies for opening a rather unnessesary issue!