Closed kloczek closed 1 week ago
I managed to reproduce this.
This happens when you compile with ENABLE_PLUGIN_LOADING=ON
, but you don't install the plugins. Then some encoders cannot be found and the tests fail.
Either installing the plugins such that the tests can find them or compiling with ENABLE_PLUGIN_LOADING=OFF
fixes this.
There is also a cmake preset "testing" exactly for this purpose (cmake .. --preset testing
).
Please make clean default ctest output.
[tkloczko@pers-jacek SPECS]$ grep %ctest *spec | wc -l
614
In all above cases it is not necessary to add any additional ctest options.
This happens when you compile with ENABLE_PLUGIN_LOADING=ON, but you don't install the plugins. Then some encoders cannot be found and the tests fail.
ctest should rely only on content of the build and source tree .. not on what is installed.
The above change should fix this. It sets the LIBHEIF_PLUGIN_PATH
environment variable to point to the plugin directory in the build tree when running ctest.
I do build with --preset=testing
[ 2s] Preset CMake variables:
[ 2s]
[ 2s] BUILD_TESTING="ON"
[ 2s] CMAKE_COMPILE_WARNING_AS_ERROR="ON"
[ 2s] ENABLE_EXPERIMENTAL_FEATURES="ON"
[ 2s] ENABLE_PLUGIN_LOADING="OFF"
[ 2s] WITH_REDUCED_VISIBILITY="OFF"
[ 2s] WITH_UNCOMPRESSED_CODEC="ON"
Even with the mentioned commit [https://github.com/strukturag/libheif/commit/74a4393dee85c09a457b961e087e5f4c1d2f001f] added, although in a different line now:
9/22 Testing: extended_type
9/22 Test: extended_type
Command: "/home/abuild/rpmbuild/BUILD/libheif-1.19.1/build/tests/extended_type"
Directory: /home/abuild/rpmbuild/BUILD/libheif-1.19.1/build/tests
"extended_type" start time: Nov 03 20:16 UTC
Output:
----------------------------------------------------------
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
extended_type is a Catch v2.13.9 host application.
Run with -? for options
-------------------------------------------------------------------------------
make extended type
-------------------------------------------------------------------------------
/home/abuild/rpmbuild/BUILD/libheif-1.19.1/tests/extended_type.cc:37
...............................................................................
/home/abuild/rpmbuild/BUILD/libheif-1.19.1/tests/extended_type.cc:51: FAILED:
REQUIRE( err.code == heif_error_Ok )
with expansion:
8 == 0
===============================================================================
test cases: 1 | 1 failed
assertions: 5 | 4 passed | 1 failed
<end of output>
Test time = 0.01 sec
----------------------------------------------------------
Test Failed.
"extended_type" end time: Nov 03 20:16 UTC
"extended_type" time elapsed: 00:00:00
----------------------------------------------------------
@arjendekorte Can you confirm that it works now with the current master branch (a5ea7ac716523dadd4d7fa408fc2b325bb005453)? Encoder availability is now checked at runtime and the test is skipped if the encoder is not available.
I'm trying to run this, but I'm running into problems with catch2-populate. This appears not to be present in the catch2-devel (3.7.1) package I installed.
Cmake should actually download the required version of catch2 automatically (current master
branch, 2339c6abcb). Doesn't it do that for you? Can you attach the full cmake output?
That won't work, as there is no connectivity in the openSUSE buildservice during build. Everything required for build has to be present before build start.
How would one then provide the dependencies to the buildservice? Say, if the build needs the .tar.gz of a specific catch2 version or a checkout of another git repository?
We have the same case in some plugins. For example, in order to compile the rav1e plugin, it is often required to run the rav1e.cmd
script in the third-party
directory to download and build rav1e if there are no prebuilt packages available.
Then either that version must be made available as a package which can be included via a BuildRequires. Currently there are two versions available.
But since there is a Catch2 version 3.7.1 available, I guess it should be possible to use that instead of downloading from git. Unfortunately I won't have time to figure that out until at least friday.
Yes, 3.7.1 should work, I was just asking out of interest how that works in principle.
I did not understand what kind of issue there is with "catch2-populate" (I don't even know what that is). Is that anything I can look for in libheif or is that something on your side?
I guess 'catch2-populate' was emitted by cmake when it failed to download from git. But I haven't looked into this deeply after I found out it was trying to do so, which is known to fail on openSUSE for a long time.
How would one then provide the dependencies to the buildservice? Say, if the build needs the .tar.gz of a specific catch2 version or a checkout of another git repository?
We have the same case in some plugins. For example, in order to compile the rav1e plugin, it is often required to run the
rav1e.cmd
script in thethird-party
directory to download and build rav1e if there are no prebuilt packages available.
It's pretty common that package build services don't have access to any outside sources besides their own package repositories to install dependencies from. The same applies for example for Debian or Ubuntu where you will have to install any dependencies from the repository but can't checkout them from git or download from somewhere. If a dependency is not available as package yet, it has to be packaged first in order to be used.
Introducing catch2 as a build dependency is one way, but in my Ubuntu 22.04 we only have catch2 version 2.13.8, which is almost 3 years old. And the situation for other distributions is probably not better.
Is there any other solution instead of reverting to including the amalgamated catch2 code in the libheif repository and much larger compilation times?
If you want to support building on older platforms and require new catch2 code, you will have to include that catch2 code in the libheif repository (if the license permits it), otherwise running the tests will have to be disabled on these platforms (which is not optimal for obvious reasons).
@arjendekorte Please try again. I reverted back to including the required catch2 with libheif. Tests with unavailable codecs should be skipped now.
That seems to do the trick:
[ 85s] Start 8: uncompressed_box
[ 85s] 8/22 Test #8: uncompressed_box .......................... Passed 0.00 sec
[ 85s] Start 9: extended_type
[ 85s] 9/22 Test #9: extended_type .............................***Skipped 0.00 sec
[ 85s] Start 10: region
[ 85s] 10/22 Test #10: region .................................... Passed 0.16 sec
Ok, thanks for testing. Can I close this now?
I'll soon do a v1.9.2 release that includes this change.
Yes, issue can be closed now. Looking forward to v1.9.2 release
Just tested 1.9.2 and still have two units failing
@kloczek Can you send the full CMake configure output so that I can see what codecs you have enabled?
I'm seeing a problem in the extend_type test too when building with X265 enabled:
[ 94s] -------------------------------------------------------------------------------
[ 94s] make extended type
[ 94s] -------------------------------------------------------------------------------
[ 94s] /home/abuild/rpmbuild/BUILD/libheif-1.19.2/tests/extended_type.cc:37
[ 94s] ...............................................................................
[ 94s]
[ 94s] /home/abuild/rpmbuild/BUILD/libheif-1.19.2/tests/extended_type.cc:50: FAILED:
[ 94s] REQUIRE( err.code == heif_error_Ok )
[ 94s] with expansion:
[ 94s] 8 == 0
Error 8 is heif_error_Encoder_plugin_error
Which x265 version do you have installed?
libx265-213-4.0~+6~git20240927.a009ec077-1.1
Source and build specification can be found here: https://build.opensuse.org/package/show/home:X0F:DEPS/x265
Note that we're not really affected by this problem in openSUSE. Due to licencing restrictions we're unable to include X265 support in the packages we build. It only made available for testing purposes.
Thank you. I'll try whether I can reproduce this with x265 4.0
@kloczek Can you send the full CMake configure output so that I can see what codecs you have enabled?
Here it is:
[tkloczko@pers-jacek x86_64-redhat-linux-gnu]$ cmake -L
CMake Warning:
No source or binary directory provided. Both will be assumed to be the
same as the current working directory, but note that this warning will
become a fatal error in future CMake releases.
CMake Error: The source directory "/home/tkloczko/rpmbuild/BUILD/libheif-1.19.2/x86_64-redhat-linux-gnu" does not appear to contain CMakeLists.txt.
Specify --help for usage, or press the help button on the CMake GUI.
-- Cache values
BROTLI_COMMON_LIB:FILEPATH=BROTLI_COMMON_LIB-NOTFOUND
BROTLI_DEC_INCLUDE_DIR:PATH=BROTLI_DEC_INCLUDE_DIR-NOTFOUND
BROTLI_DEC_LIB:FILEPATH=BROTLI_DEC_LIB-NOTFOUND
BROTLI_ENC_INCLUDE_DIR:PATH=BROTLI_ENC_INCLUDE_DIR-NOTFOUND
BROTLI_ENC_LIB:FILEPATH=BROTLI_ENC_LIB-NOTFOUND
BUILD_SHARED_LIBS:BOOL=ON
BUILD_TESTING:BOOL=ON
CMAKE_BUILD_TYPE:STRING=RelWithDebInfo
CMAKE_COMPILE_WARNING_AS_ERROR:BOOL=ON
CMAKE_INSTALL_PREFIX:PATH=/usr
ENABLE_COVERAGE:BOOL=OFF
ENABLE_EXPERIMENTAL_FEATURES:BOOL=OFF
ENABLE_EXPERIMENTAL_MINI_FORMAT:BOOL=OFF
ENABLE_MULTITHREADING_SUPPORT:BOOL=ON
ENABLE_PARALLEL_TILE_DECODING:BOOL=ON
ENABLE_PLUGIN_LOADING:BOOL=ON
FUZZING_COMPILE_OPTIONS:STRING=-fsanitize=fuzzer,address,shift,integer -fno-sanitize-recover=shift,integer
FUZZING_CXX_COMPILER:STRING=clang++
FUZZING_C_COMPILER:STRING=clang
FUZZING_LINKER_OPTIONS:STRING=
PLUGIN_DIRECTORY:STRING=/usr/lib64/libheif
PLUGIN_INSTALL_DIRECTORY:STRING=
Tiff_DIR:PATH=Tiff_DIR-NOTFOUND
WITH_AOM_DECODER:BOOL=ON
WITH_AOM_DECODER_PLUGIN:BOOL=ON
WITH_AOM_ENCODER:BOOL=ON
WITH_AOM_ENCODER_PLUGIN:BOOL=ON
WITH_DAV1D:BOOL=OFF
WITH_DAV1D_PLUGIN:BOOL=ON
WITH_EXAMPLES:BOOL=ON
WITH_FFMPEG_DECODER:BOOL=OFF
WITH_FFMPEG_DECODER_PLUGIN:BOOL=ON
WITH_FUZZERS:BOOL=OFF
WITH_GDK_PIXBUF:BOOL=ON
WITH_HEADER_COMPRESSION:BOOL=OFF
WITH_JPEG_DECODER:BOOL=OFF
WITH_JPEG_DECODER_PLUGIN:BOOL=ON
WITH_JPEG_ENCODER:BOOL=OFF
WITH_JPEG_ENCODER_PLUGIN:BOOL=ON
WITH_KVAZAAR:BOOL=OFF
WITH_KVAZAAR_PLUGIN:BOOL=ON
WITH_LIBDE265:BOOL=ON
WITH_LIBDE265_PLUGIN:BOOL=ON
WITH_LIBSHARPYUV:BOOL=ON
WITH_OPENJPH_ENCODER:BOOL=OFF
WITH_OPENJPH_ENCODER_PLUGIN:BOOL=ON
WITH_OpenH264_DECODER:BOOL=ON
WITH_OpenH264_DECODER_PLUGIN:BOOL=OFF
WITH_OpenJPEG_DECODER:BOOL=OFF
WITH_OpenJPEG_DECODER_PLUGIN:BOOL=ON
WITH_OpenJPEG_ENCODER:BOOL=OFF
WITH_OpenJPEG_ENCODER_PLUGIN:BOOL=ON
WITH_RAV1E:BOOL=OFF
WITH_RAV1E_PLUGIN:BOOL=ON
WITH_REDUCED_VISIBILITY:BOOL=ON
WITH_SvtEnc:BOOL=OFF
WITH_SvtEnc_PLUGIN:BOOL=ON
WITH_UNCOMPRESSED_CODEC:BOOL=ON
WITH_UVG266:BOOL=OFF
WITH_UVG266_PLUGIN:BOOL=OFF
WITH_VVDEC:BOOL=OFF
WITH_VVDEC_PLUGIN:BOOL=OFF
WITH_VVENC:BOOL=OFF
WITH_VVENC_PLUGIN:BOOL=OFF
WITH_X265:BOOL=ON
WITH_X265_PLUGIN:BOOL=ON
Error 8 is
heif_error_Encoder_plugin_error
Which x265 version do you have installed?
3.5
I've tried with the same x265 version and still cannot reproduce the error.
My above commit shows the whole error message when the extended_type
test fails. Could you please rerun the test with the above commit and show me the output?
After applying the latest commit (with changes so that it cleanly applies to 1.19.2) I get the following output:
[ 84s] Start 9: extended_type
[ 84s] 9/22 Test #9: extended_type .............................***Failed 0.01 sec
[ 84s] Randomness seeded to: 2729141873
[ 84s]
[ 84s] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[ 84s] extended_type is a Catch2 v3.7.1 host application.
[ 84s] Run with -? for options
[ 84s]
[ 84s] -------------------------------------------------------------------------------
[ 84s] make extended type
[ 84s] -------------------------------------------------------------------------------
[ 84s] /home/abuild/rpmbuild/BUILD/libheif-1.19.2/tests/extended_type.cc:37
[ 84s] ...............................................................................
[ 84s]
[ 84s] /home/abuild/rpmbuild/BUILD/libheif-1.19.2/tests/extended_type.cc:51: FAILED:
[ 84s] REQUIRE( err.code == heif_error_Ok )
[ 84s] with expansion:
[ 84s] 8 == 0
[ 84s] with message:
[ 84s] heif_context_encode_image: Encoder plugin generated an error: Unsupported bit
[ 84s] depth: Bit depth not supported by x265
[ 84s]
[ 84s] ===============================================================================
[ 84s] test cases: 1 | 1 failed
[ 84s] assertions: 4 | 3 passed | 1 failed
Ok, now I understand what goes wrong. x265 comes in two variants, one for encoding 8bit images and one for 10bit images. Usually, every installation supports 8bit and the 10bit support is optional.
In the libheif test, it is trying to get the 8bit x265 interface. Apparently, your x265 binary can only support 10bit (or maybe it includes no encoder at all).
In any case, this looks like a broken x265 install since it does not handle the most common case of 8 bit images. It seems valid to me that the test fails in this case.
I can confirm that the problem (in my case) is indeed x265. The version I build against only supports 12bit. When build with support for 8-, 10- aand 12-bit the test indeed passes. As far as I'm concerned this issue can be closed.
@arjendekorte Thank you for confirming the root cause.
cmake settings:
and test suite fails in 3 units: