[Issue]: unit tests are not validating results

AngryLoki commented 6 months ago

Problem Description

Hi, I'm writing ebuild for rpp, which also includes running tests.

I run uniqueFunctionalities_hip at this moment (https://github.com/AngryLoki/gentoo/blob/rocm-6.0.2/sci-libs/rpp/rpp-6.0.2.ebuild#L85). However there is a major flaw, not in a library, but in a test.

Right now is what I see in the output (among other seemingly fine results):

Input:
255.00  254.00  253.00  252.00  251.00  250.00  249.00  248.00  247.00  246.00  245.00  244.00  130.00  129.00  128.00  127.00  126.00  125.00  124.00  123.00
122.00  121.00  120.00  119.00  5.00    4.00    3.00    2.00    1.00    0.00    27.00   26.00   25.00   24.00   23.00   22.00   21.00   20.00   19.00
18.00   17.00   16.00   55.00   54.00   53.00   52.00   51.00   50.00   49.00   48.00   47.00   46.00   45.00   44.00   115.00  114.00  113.00  112.00
111.00  110.00  240.00  239.00  238.00  237.00  236.00  235.00  234.00  233.00  232.00  231.00  230.00  229.00  200.00  199.00  198.00  197.00  196.00
195.00  194.00  193.00  192.00  191.00  190.00  189.00  140.00  139.00  138.00  137.00  136.00  135.00  70.00   69.00   68.00   67.00   66.00   65.00
64.00   63.00   62.00   61.00   60.00   59.00   170.00  169.00  168.00  167.00  166.00  165.00  164.00  163.00  162.00  161.00  160.00  159.00  15.00
14.00   13.00   12.00   11.00   10.00

Output of transpose_f16:
0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00
0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00
0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00
0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00
0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00
0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00
0.00    0.00    0.00    0.00    0.00

Input:
255.00  254.00  253.00  252.00  251.00  250.00  249.00  248.00  247.00  246.00  245.00  244.00  130.00  129.00  128.00  127.00  126.00  125.00  124.00  123.00
122.00  121.00  120.00  119.00  5.00    4.00    3.00    2.00    1.00    0.00    27.00   26.00   25.00   24.00   23.00   22.00   21.00   20.00   19.00
18.00   17.00   16.00   55.00   54.00   53.00   52.00   51.00   50.00   49.00   48.00   47.00   46.00   45.00   44.00   115.00  114.00  113.00  112.00
111.00  110.00  240.00  239.00  238.00  237.00  236.00  235.00  234.00  233.00  232.00  231.00  230.00  229.00  200.00  199.00  198.00  197.00  196.00
195.00  194.00  193.00  192.00  191.00  190.00  189.00  140.00  139.00  138.00  137.00  136.00  135.00  70.00   69.00   68.00   67.00   66.00   65.00
64.00   63.00   62.00   61.00   60.00   59.00   170.00  169.00  168.00  167.00  166.00  165.00  164.00  163.00  162.00  161.00  160.00  159.00  15.00
14.00   13.00   12.00   11.00   10.00

Output of transpose_f32:
255.00  252.00  249.00  246.00  0.00    127.00  124.00  121.00  5.00    0.00    27.00   24.00   21.00   18.00   8495497217120164957564929092616192.00   52.00   49.00   46.00115.00  0.00
240.00  237.00  234.00  231.00  9046043814911473556460266810507264.00   126.00  123.00  120.00  4.00    0.00    26.00   23.00   20.00   17.00   9045905165627073593869478108594176.00        51.00   48.00   45.00   114.00
0.00    239.00  236.00  233.00  230.00  8439621555506980033477082221641728.00   125.00  122.00  119.00  3.00    0.00    25.00   22.00   19.00   16.00   9045806130423930763447486178656256.00        50.00   47.00   44.00
113.00  0.00    238.00  235.00  232.00  229.00  9045895262106759310827278915600384.00   197.00  194.00  191.00  140.00  0.00    70.00   67.00   64.00   61.00   0.00    167.00       164.00
161.00  15.00   0.00    -0.37   -0.37   -0.37   -0.37   9046053718431787839502466003501056.00   196.00  193.00  190.00  139.00  0.00    69.00   66.00   63.00   60.00   9046132946594302103840059547451392.00        166.00
163.00  160.00  14.00   0.00    -0.37   -0.37   -0.37   -0.37   9046192367716187802093254705414144.00   195.00  192.00  189.00  138.00  0.00    68.00   65.00   62.00   59.00-0.00
165.00  162.00  159.00  13.00   0.00

test_case 8 supports only 0 ip_bitDepth
ip_bitDepth = 0
test_case = 8---PASS---
---PASS--

(test exits with code 0)

There are also other tests, which produce a directory with images with various filters applied, but again I don't see how I can use these tests to automatically validate, that library actually works. Also it outputs a wall of "not implemented", which was already mentioned in https://github.com/ROCm/rpp/issues/282

May I ask to provide 2 things in rpp: 1) There should be at least some test, that fully validates the result? 2) The test application should return non-zero code, if any subtest fails (maybe use gtest).

Operating System

Gentoo

CPU

Ryzen 9 7950X3D

GPU

Other

No response

ROCm Version

ROCm 6.0.0

ROCm Component

rpp

Steps to Reproduce

No response

(Optional for Linux users) Output of /opt/rocm/bin/rocminfo --support

No response

Additional Information

No response

kiritigowda commented 5 months ago

@r-abishek can you take a look at this?

r-abishek commented 3 months ago

Please use the latest utilities/test_suite folder for running currently supported functionalities in unit test mode / qa mode / performance test mode - https://github.com/ROCm/rpp/blob/develop/utilities/test_suite/README.md

kiritigowda commented 3 months ago

@AngryLoki -- new test suite should fix this issue. Please reopen if you see any gaps. Thanks!

ROCm / rpp