search

GPUOpen-LibrariesAndSDKs / RadeonImageFilter

https://gpuopen.com/radeon-prorender-suite/
Other
48 stars 14 forks source link

AI upscale filter fail with gfx803 + MIOpen 2.0.5 #13

Closed Umio-Yasuno closed 3 years ago

Umio-Yasuno commented 3 years ago

AI upscale filter fail for error of MIOpen when use Fast model.

[MSG  @RIF]: Loaded RIF API version: 1.7.0.0xbbcfd912
INFO: MIOpen version: 2.0.5
[MSG  @RIF]: Loading model: ../RadeonImageFilter/models//upscale2x_fast.pb
INFO: Memory: required: 66355200, single node max: 58982400, requested sum: 239616000
MIOpen Error: /home/admin/JN/WS/MIOpen_Build/src/ocl/convolutionocl.cpp:1017: Fwd Convolution cannot be executed due to incorrect params
ERROR: Graph/MemNet/MemBlock_1/ResBlock_1/Conv1/Conv2D:Conv2d: miopenFindConvolutionForwardAlgorithm failed, status: miopenStatusUnknownError (7)
[ERROR@RIF]: rmlPrepareModel failed: Graph/MemNet/MemBlock_1/ResBlock_1/Conv1/Conv2D:Conv2d: miopenFindConvolutionForwardAlgorithm failed, status: miopenStatusUnknownError (7) at file /home/admin/JN/WS/RadeonProImageProcessor_Build/src/RadeonImageFilters.cpp line 1220

Also Good and Best model failed, but so VRAM of 4GB may was not enough.

OS: Debian GNU/Linux bullseye/sid CPU: Ryzen 5 2600 GPU: Radeon RX 560 4GB (Polaris11, gfx803) OpenCL: OpenCL 2.0 AMD-APP (3212.0)

Log

$ MIOPEN_LOG_LEVEL=7 ./rif-cmd -i ../input/sample.png -o ../output/debug.png -f ai_upscale "3,../RadeonImageFilter/models/" (1: Good, 2: Best, 3: Fast)

fast_model.log fast_model_rif-trace-1616223509-1.7.0.0xbbcfd912.tar.gz

good_model.log good_model_rif-trace-1616224368-1.7.0.0xbbcfd912.tar.gz

best_model.log best_model_rif-trace-1616224280-1.7.0.0xbbcfd912.tar.gz

stmuxa commented 3 years ago

Hey @Umio-Yasuno Could you please try updated version? Make sure you replace files in models/kernels directory. https://drive.google.com/file/d/1i0MLLN2pY5DPcHcz-TwDDEvO3JhHdmGq/view?usp=sharing

Umio-Yasuno commented 3 years ago

Thanks @stmuxa However, the result was the same. The log did not change either.

20210409.log

Backend API:     OpenCL
Vendor:      Advanced Micro Devices, Inc.
Device Name:     Baffin [Radeon RX 550 640SP / RX 560/560X]

[MSG  @RIF]: Loaded RIF API version: 1.7.0.0xbbcfd912
INFO: MIOpen version: 2.0.5
../../../src/new-miopen/radeonimagefilters-move-miopen-to-2.10.0-linux-4171ca-Ubuntu18-dynamic/models/kernels
Kernel Source Dir: ../../../src/new-miopen/radeonimagefilters-move-miopen-to-2.10.0-linux-4171ca-Ubuntu18-dynamic/models/kernels/
Kernel Cache  Dir: kernelsCache
Execute
[MSG  @RIF]: Loading model: ../../../src/new-miopen/radeonimagefilters-move-miopen-to-2.10.0-linux-4171ca-Ubuntu18-dynamic/models/upscale2x_fast.pb
INFO: Memory: required: 66355200, single node max: 58982400, requested sum: 239616000
MIOpen Error: /home/admin/JN/WS/MIOpen_Build/src/ocl/convolutionocl.cpp:1017: Fwd Convolution cannot be executed due to incorrect params
ERROR: Graph/MemNet/MemBlock_1/ResBlock_1/Conv1/Conv2D:Conv2d: miopenFindConvolutionForwardAlgorithm failed, status: miopenStatusUnknownError (7)
[ERROR@RIF]: rmlPrepareModel failed: Graph/MemNet/MemBlock_1/ResBlock_1/Conv1/Conv2D:Conv2d: miopenFindConvolutionForwardAlgorithm failed, status: miopenStatusUnknownError (7) at file /home/admin/JN/WS/RadeonProImageProcessor_Build/src/RadeonImageFilters.cpp line 1220
givinar commented 3 years ago

Hi @Umio-Yasuno We found the reasone of issue. It'll be fixed in the next version of RIF. Thanks.

Umio-Yasuno commented 3 years ago

I tried v1.7.1 (Ubuntu18-Clang and Ubuntu20), but that failed.

OS: Debian GNU/Linux 11 (bullseye) CPU: Ryzen 5 2600

GPU: Radeon RX 560 4GB (Polaris11, gfx803) OpenCL: OpenCL 2.0 AMD-APP (3275.0)

fast_model.log rif-trace-1623078002-1.7.1.0xfe53c43a.tar.gz

givinar commented 3 years ago

Hi @Umio-Yasuno, Are you sure you are using the correct version of libRadeonML and libRadeonML_MIOpen? It should be 0.9.12. Also make sure the softlinks point to the correct files. If so, could you send libMIOpen, libRadeonML and libRadeonML_MIOpen? Thanks.

Umio-Yasuno commented 3 years ago

I am using RadeoImageFilter as a submodule and setting LD_LIBRARY_PATH="${HOME}/github/rif-cmd/RadeonImageFilter/Ubuntu18-Clang/Dynamic/":$LD_LIBRARY_PATH for .bashrc. https://github.com/Umio-Yasuno/rif-cmd

Is there a problem with my CMakeLists.txt? https://github.com/Umio-Yasuno/rif-cmd/blob/main/CMakeLists.txt

I checked with tree command, and the soft link seems to be correct.

RadeonImageFilter/Ubuntu18-Clang/
└── Dynamic
    ├── libMIOpen.so -> libMIOpen.so.2
    ├── libMIOpen.so.2 -> libMIOpen.so.2.0.5
    ├── libMIOpen.so.2.0.5
    ├── libOpenImageDenoise.so -> libOpenImageDenoise.so.0
    ├── libOpenImageDenoise.so.0 -> libOpenImageDenoise.so.0.9.0
    ├── libOpenImageDenoise.so.0.9.0
    ├── libRadeonImageFilters.so -> libRadeonImageFilters.so.1
    ├── libRadeonImageFilters.so.1 -> libRadeonImageFilters.so.1.7.1
    ├── libRadeonImageFilters.so.1.7.1
    ├── libRadeonML_MIOpen.so -> libRadeonML_MIOpen.so.0
    ├── libRadeonML_MIOpen.so.0 -> libRadeonML_MIOpen.so.0.9.12
    ├── libRadeonML_MIOpen.so.0.9.12
    ├── libRadeonML.so -> libRadeonML.so.0
    ├── libRadeonML.so.0 -> libRadeonML.so.0.9.12
    └── libRadeonML.so.0.9.12
RadeonImageFilter/Ubuntu20/
└── Dynamic
    ├── libMIOpen.so -> libMIOpen.so.2
    ├── libMIOpen.so.2 -> libMIOpen.so.2.0.5
    ├── libMIOpen.so.2.0.5
    ├── libOpenImageDenoise.so -> libOpenImageDenoise.so.0
    ├── libOpenImageDenoise.so.0 -> libOpenImageDenoise.so.0.9.0
    ├── libOpenImageDenoise.so.0.9.0
    ├── libRadeonImageFilters.so -> libRadeonImageFilters.so.1
    ├── libRadeonImageFilters.so.1 -> libRadeonImageFilters.so.1.7.1
    ├── libRadeonImageFilters.so.1.7.1
    ├── libRadeonML_MIOpen.so -> libRadeonML_MIOpen.so.0
    ├── libRadeonML_MIOpen.so.0 -> libRadeonML_MIOpen.so.0.9.12
    ├── libRadeonML_MIOpen.so.0.9.12
    ├── libRadeonML.so -> libRadeonML.so.0
    ├── libRadeonML.so.0 -> libRadeonML.so.0.9.12
    └── libRadeonML.so.0.9.12

2 directories, 30 files
givinar commented 3 years ago

Thanks @Umio-Yasuno I checked on my side, everything works fine. But I saw this line in your log - MIOpen(OpenCL): Info [DetectAmdRocmMetadataVersion] ROCm MD version AMDHSA_1_0, MIOpen version 2.0.5.2d225393-dirty What driver are you using? The RIF shouldn't work on ROCm, use amdgpu pro instead. It can be the reason .

Umio-Yasuno commented 3 years ago

I am using ROCm 4.2 (rocm-opencl: 2.0.0.40200-21). However, the other filters for OpenCL only without MIOpen/RadeonML work fine. clinfo.txt

I tried to install AMD GPU-PRO but it failed due to version. I am currently in a difficult position to build a new environment for verification. Sorry.

BenjaminCoquelle commented 3 years ago

We don't support ROCm, only amdgpu-pro stack That's why we ship a custom version of miopen here that can run on amdgpu-pro I will actually close that issue as it is on a non supported system

Umio-Yasuno commented 3 years ago

Do you have any plans to support it in the ROCm OpenCL Driver in the future?

BenjaminCoquelle commented 3 years ago

No we don't as the main goal of the library is to provide support for amdgpu-pro stack on linux. The lib is mainly aimed to be integrated onto graphics based application and ROCm is not designed for that

Umio-Yasuno commented 3 years ago

I understand. Thank you for your support! @BenjaminCoquelle @givinar @stmuxa