core: Failed Core_Xor/ElemWiseTest.accuracy in debug build (5.x)

Kumataro commented 8 months ago

System Information

OpenCV version: 5.x Operating System / Platform: Ubuntu 23.10 (x86-64) Compiler & compiler version: GCC 13.2.0 (This problem occurs in debug build only)

cmake -S opencv5 -B build5-main_debug -GNinja -DCMAKE_BUILD_TYPE=debug

-- General configuration for OpenCV 5.0.0-pre =====================================
--   Version control:               4.8.0-578-g788c7252dc
--
--   Platform:
--     Timestamp:                   2024-01-21T00:38:11Z
--     Host:                        Linux 6.5.0-14-generic x86_64
--     CMake:                       3.27.4
--     CMake generator:             Ninja
--     CMake build tool:            /usr/bin/ninja
--     Configuration:               debug

Detailed description

opencv_test_core is failed at Core_Xor and Core_XorS with CV_16F. It causes only debug build, not release build. To remove bitwise operations for CV_16F are one of solutions, but to decide it is need to more comment.

Note: Google Test filter = Core_Xor/E*
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from Core_Xor/ElemWiseTest
[ RUN      ] Core_Xor/ElemWiseTest.accuracy/0, where GetParam() = 16-byte object <F0-33 31-9A 9E-55 00-00 A0-9E 34-9A 9E-55 00-00>
/home/kmtr/work/opencv5/modules/core/test/test_arithm.cpp:1611: Failure
too big relative difference (inf > 0) between 16fC3 4-dim (12228 x 1 x 1 x 1) 'dst0' and 'dst' at [3107;
 0;
 0;
 0].
- dst0:
[-nan].
- dst:
[-nan].

src[0] ~ 16fC3 4-dim (12228 x 1 x 1 x 1)
testCase #920

[  FAILED  ] Core_Xor/ElemWiseTest.accuracy/0, where GetParam() = 16-byte object <F0-33 31-9A 9E-55 00-00 A0-9E 34-9A 9E-55 00-00> (2370 ms)
[----------] 1 test from Core_Xor/ElemWiseTest (2370 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (2372 ms total)
[  PASSED  ] 0 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] Core_Xor/ElemWiseTest.accuracy/0, where GetParam() = 16-byte object <F0-33 31-9A 9E-55 00-00 A0-9E 34-9A 9E-55 00-00>

 1 FAILED TEST

Steps to reproduce

git clone opencv 5.x
cmake -S opencv5 -B build5-main_debug -GNinja -DCMAKE_BUILD_TYPE=debug
cmake --build build5-main_debug
cd build5-main_debug
./bin/opencv_test_core --gtest_filter="Core_Xor/E*"

This nan values in dst0 and dst are not same.

kmtr@kmtr-None:~/work/build5-main_debug$ ./bin/opencv_test_core --gtest_filter="Core_Xor/E*"
[ INFO:0@0.127] global ocl.cpp:1185 haveOpenCL Initialize OpenCL runtime...
[ INFO:0@0.127] global ocl.cpp:1191 haveOpenCL OpenCL: found 0 platforms
CTEST_FULL_OUTPUT
OpenCV version: 5.0.0-pre
OpenCV VCS version: 4.8.0-578-g788c7252dc
Build type: debug
WARNING: build value differs from runtime: Debug
Compiler: /usr/bin/c++  (ver 13.2.0)
[ INFO:0@0.150] global registry_parallel.impl.hpp:96 ParallelBackendRegistry core(parallel): Enabled backends(3, sorted by priority): ONETBB(1000); TBB(990); OPENMP(980)
Parallel framework: pthreads (nthreads=8)
CPU features: SSE SSE2 SSE3 *SSE4.1 *SSE4.2 *FP16 *AVX *AVX2 *AVX512-SKX?
Intel(R) IPP version: ippIP AVX2 (l9) 2021.10.0 (-) Sep 18 2023
Intel(R) IPP features code: 0x8000
OpenCL is disabled
TEST: Skip tests with tags: 'mem_6gb', 'verylong', 'debug_verylong'
Note: Google Test filter = Core_Xor/E*
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from Core_Xor/ElemWiseTest
[ RUN      ] Core_Xor/ElemWiseTest.accuracy/0, where GetParam() = 16-byte object <F0-33 31-9A 9E-55 00-00 A0-9E 34-9A 9E-55 00-00>
/home/kmtr/work/opencv5/modules/core/test/test_arithm.cpp:1611: Failure
too big relative difference (inf > 0) between 16fC3 4-dim (12228 x 1 x 1 x 1) 'dst0' and 'dst' at [3107;
 0;
 0;
 0].
- dst0:
[-nan].
- dst:
[-nan].

src[0] ~ 16fC3 4-dim (12228 x 1 x 1 x 1)
testCase #920

[  FAILED  ] Core_Xor/ElemWiseTest.accuracy/0, where GetParam() = 16-byte object <F0-33 31-9A 9E-55 00-00 A0-9E 34-9A 9E-55 00-00> (2370 ms)
[----------] 1 test from Core_Xor/ElemWiseTest (2370 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (2372 ms total)
[  PASSED  ] 0 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] Core_Xor/ElemWiseTest.accuracy/0, where GetParam() = 16-byte object <F0-33 31-9A 9E-55 00-00 A0-9E 34-9A 9E-55 00-00>

 1 FAILED TEST
kmtr@kmtr-None:~/work/build5-main_debug$

Issue submission checklist

[X] I report the issue, it's not a question
[X] I checked the problem with documentation, FAQ, open issues, forum.opencv.org, Stack Overflow, etc and have not found any solution
[X] I updated to the latest OpenCV version and the issue is still there
[X] There is reproducer code and related data files (videos, images, onnx, etc)

Kumataro commented 8 months ago

In release build (no errors).

kmtr@kmtr-None:~/work/build5-main_debug$ ./bin/opencv_test_core --gtest_filter="Core_Xor/E*"
CTEST_FULL_OUTPUT
OpenCV version: 5.0.0-pre
OpenCV VCS version: 4.8.0-578-g788c7252dc
Build type: release
WARNING: build value differs from runtime: Release
Compiler: /usr/bin/c++  (ver 13.2.0)
Parallel framework: pthreads (nthreads=8)
CPU features: SSE SSE2 SSE3 *SSE4.1 *SSE4.2 *FP16 *AVX *AVX2 *AVX512-SKX?
Intel(R) IPP version: ippIP AVX2 (l9) 2021.10.0 (-) Sep 18 2023
Intel(R) IPP features code: 0x8000
OpenCL is disabled
TEST: Skip tests with tags: 'mem_6gb', 'verylong'
Note: Google Test filter = Core_Xor/E*
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from Core_Xor/ElemWiseTest
[ RUN      ] Core_Xor/ElemWiseTest.accuracy/0, where GetParam() = 16-byte object <F0-73 70-6C C4-55 00-00 A0-DE 73-6C C4-55 00-00>
[       OK ] Core_Xor/ElemWiseTest.accuracy/0 (1014 ms)
[----------] 1 test from Core_Xor/ElemWiseTest (1014 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (1015 ms total)
[  PASSED  ] 1 test.
kmtr@kmtr-None:~/work/build5-main_debug$

Kumataro commented 8 months ago

I think testing mat types are defined with following code. DEPTH_MASK_ALL_BUT_8S seems to contain DEPTH_MASK_16F after OpenCV5. So some matrix with CV_16F is tested for Xor/XorS operations.

https://github.com/opencv/opencv/blob/788c7252dcf3f605f95ded7db698f8a46d83cacd/modules/core/test/test_arithm.cpp#L37-L41

OpenCV4 https://github.com/opencv/opencv/blob/e64857c5611d5898b7b30640a775331488a5ebef/modules/core/include/opencv2/core/mat.hpp#L308-L312

OpenCV5 https://github.com/opencv/opencv/blob/788c7252dcf3f605f95ded7db698f8a46d83cacd/modules/core/include/opencv2/core/mat.hpp#L302-L306

opencv / opencv