AI denoise filter creation error on macOS with Radeon Pro 5300M

[deranen]

Hello,

I am getting a RIF_ERROR_INTERNAL_ERROR return value when calling rifContextCreateImageFilter with a filter of type RIF_IMAGE_FILTER_AI_DENOISE. This is happening on a 2019 Macbook Pro with a Radeon Pro 5300M GPU running macOS Catalina 10.15.6. The Radeon Image Filter version is 1.6.1.

Is the GPU supported? My code is picking the correct GPU (skipping the integrated Intel GPU). All calls up to this point work and this same code also works on Windows computers with AMD cards. The AMD GPU has 4 GB of RAM which should be enough, and if it isn't I would expect to get an out-of-memory error.

Ideas?

[BenjaminCoquelle]

Hello

Can you enable the log and see what message we also return. I have the feeling the libRadeonML_MPS can't be found and loaded and in that case you may want to add it to your path. But the log will tell us

To that matter please set the following environment variable RIF_LOG_LEVEL=3

[deranen]

I tried setting the environment variable in my project settings, but I don't get any output at least when testing this on a working GPU in Windows. Should log level 3 be outputting text even though there are no errors? Should the text be going to the output window in Visual Studio?

By the way, I'm doing this in a C# project which links and wraps the libRadeonImageFilters.dylib library file. Will doing it this way prevent debugging output from the C++ libraries?

[BenjaminCoquelle]

No you won't see it in the output window of Visual Studio, we only return in stdout and stderr Here an example of the error in such case [ERROR@RIF]: Loading RadeonML failed with error: -1 at file D:\dev\git\RIF1.6.1\RadeonProImageProcessing\src\AIFilters.cpp line 421

Though with that level of LOG you will only see ERROR and WARNING.

Do you at least see this on your side [MSG @RIF]: Loaded RIF API version: 1.6.1.0xe4886697 At context creation we give that info all the time This is just to check you can see the logs

[deranen]

Hi @BenjaminCoquelle,

Thank you for your help! I am now able to get error messages from the computer having these issues. Here it is:

[MSG @RIF]: Loaded RIF API version: 1.6.1.0xe4886697
[ERROR@RIF]: Loading RadeonML failed with error: -2 at file /Users/admin/JN/WS/RadeonProImageProcessor_Build/src/AIFilters.cpp line 389

Any ideas?

[BenjaminCoquelle]

It seems you requested a OpenCL backend, am I correct On OSX we only support metal

We need to improve the error and feedback for the user in that case

[deranen]

Silly me. I changed to using Metal. I now get "OUT_OF_VIDEO_MEMORY" error. The GPU has 4 GB of memory, but I guess not all of it is free.

Thanks again for the help. I'm hoping the memory usage requirements will decrease a bit so that we can use a 4 GB RAM GPU.

Full output:

[MSG  @RIF]: Loaded RIF API version: 1.6.1.0xe4886697
INFO: Using Metal device: AMD Radeon Pro 5300M
[MSG  @RIF]: Loading model: /Users/<user>/dev/AMDDenoiser/bin/OSX/Debug/models/denoise_c3_hdr_gamma_2.5.pb
[ERROR@RIF]: RIF_ERROR_OUT_OF_VIDEO_MEMORY at file /Users/admin/JN/WS/RadeonProImageProcessor_Build/src/AIFilters.cpp line 620

[BenjaminCoquelle]

Can you try with FP16? That should hopefully pass in your case

[deranen]

Ah yes. You are right, doing ImageSetComputeType to Half makes things not give out-of-memory errors. Unfortunately, the denoiser output is glitchy. This might be because of something we're doing wrong on our side, of course.

It should still be fine to give input and output textures as Float32, right?

Output:

[MSG @RIF]: Loaded RIF API version: 1.6.1.0xe4886697
INFO: Using Metal device: AMD Radeon Pro 5300M
[MSG @RIF]: Loading model: /Users/<user>/dev/AMDDenoiser/bin/OSX/Debug/models/denoise_c3_hdr_gamma_2.5_f16.onnx
INFO: Model info: 
 domain: 
 ir_version: 6
 producer_name: AMD
 producer_version: 
 version: 0
 opset domain: opset version11

Glitchy output:

It looks like we're reading the output buffer before the denoising has finished. I added rifFlushQueue followed by a rifSyncronizeQueue, but it didn't help.

We will continue trying to figure this out. Thanks for the help!

[BenjaminCoquelle]

ok we will check on that internally

[deranen]

@BenjaminCoquelle Any news on this? Can you run the AI denoiser on a similar GPU on macOS, or are you getting the same results as me?

[BenjaminCoquelle]

Hi @deranen, sorry for the late answer. By any chance do you have a system running BigSur, that bug seems to be a driver bug in Catalina which is fixed in BigSur

[deranen]

@BenjaminCoquelle Thanks. After updating to Big Sur the blocky output is gone, but instead the output is most often completely white, and the other times one can see the denoised output. It looks like we're getting closer to having this work, but something is still amiss...

[BenjaminCoquelle]

can you make a RIF trace so we can debug together https://radeon-pro.github.io/RadeonProRenderDocs/en/rif/tracing.html

[deranen]

Here's a RIF trace, and apologies in advance for the file size. There's 72 frames of data and it compressed to 100MB: https://drive.google.com/file/d/1Dpeqa1pCsKDuAzXNhUneqkm1WEWCwyqL/view?usp=sharing

It seems like enabling RIF tracing affected the output of the denoiser. While tracing the output was all black.

[BenjaminCoquelle]

@deranen just to let you know we are still working on this issue and we haven't forgotten about you. We have a lead for that problem