Build breaks (in Arch Linux) because of removal of Butteraugli from libjxl API

[rokibhasansagar]

They recently (June 20) merged https://github.com/libjxl/libjxl/pull/2576 on libjxl. So, at first glance, <jxl/butteraugli.h> seems to be not needed to include in Endless_Merging branch (and maybe others too), But that's actually not the case (detail is on follow-up comment). https://github.com/Clybius/aom-av1-lavish/blob/83500ccf39c98e4688acda163f87c12d8b1a854a/aom_dsp/butteraugli.h#L16

It is breaking the build of aom-av1-lavish as of now (Tested in Arch Docker with AUR).

aom_dsp/butteraugli.c.o build error log

``` [92/499] Building C object CMakeFiles/aom_dsp_encoder.dir/aom_dsp/butteraugli.c.o FAILED: CMakeFiles/aom_dsp_encoder.dir/aom_dsp/butteraugli.c.o /usr/bin/cc -I/home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish -I/home/app/.cache/paru/clone/aom-av1-lavish-git/src/build -I/home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/apps -I/home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/common -I/home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/examples -I/home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/stats -I/home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/third_party/libyuv/include -I/usr/include/libvmaf -I/home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/third_party/libwebm -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -O3 -DNDEBUG -std=c99 -Wall -Wdisabled-optimization -Wextra -Wfloat-conversion -Wformat=2 -Wimplicit-function-declaration -Wlogical-op -Wpointer-arith -Wsign-compare -Wtype-limits -Wuninitialized -Wunused -Wvla -Wstack-usage=100000 -Wshadow -Wundef -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/libvmaf -DHWY_SHARED_DEFINE -fPIC -MD -MT CMakeFiles/aom_dsp_encoder.dir/aom_dsp/butteraugli.c.o -MF CMakeFiles/aom_dsp_encoder.dir/aom_dsp/butteraugli.c.o.d -o CMakeFiles/aom_dsp_encoder.dir/aom_dsp/butteraugli.c.o -c /home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/aom_dsp/butteraugli.c In file included from /home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/aom_dsp/butteraugli.c:14: /home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/aom_dsp/butteraugli.h:16:10: fatal error: jxl/butteraugli.h: No such file or directory 16 | #include | ^~~~~~~~~~~~~~~~~~~ compilation terminated. ```

There might be more build errors if the line is removed, but I haven't tested anything.

Please correct the errors to match the libjxl upstream.

[rokibhasansagar]

Okay, Removing the line as is is a No-Go. It will still not fix broken build of aom_dsp/butteraugli.c.o as the codes related to JXL in this repository still needs the header file.

Error Details after removing the inclusion of the said header file

``` [97/499] Building C object CMakeFiles/aom_dsp_encoder.dir/aom_dsp/butteraugli.c.o FAILED: CMakeFiles/aom_dsp_encoder.dir/aom_dsp/butteraugli.c.o /usr/bin/cc -I/home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish -I/home/app/.cache/paru/clone/aom-av1-lavish-git/src/build -I/home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/apps -I/home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/common -I/home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/examples -I/home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/stats -I/home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/third_party/libyuv/include -I/usr/include/libvmaf -I/home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/third_party/libwebm -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -O3 -DNDEBUG -std=c99 -Wall -Wdisabled-optimization -Wextra -Wfloat-conversion -Wformat=2 -Wimplicit-function-declaration -Wlogical-op -Wpointer-arith -Wsign-compare -Wtype-limits -Wuninitialized -Wunused -Wvla -Wstack-usage=100000 -Wshadow -Wundef -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/libvmaf -DHWY_SHARED_DEFINE -fPIC -MD -MT CMakeFiles/aom_dsp_encoder.dir/aom_dsp/butteraugli.c.o -MF CMakeFiles/aom_dsp_encoder.dir/aom_dsp/butteraugli.c.o.d -o CMakeFiles/aom_dsp_encoder.dir/aom_dsp/butteraugli.c.o -c /home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/aom_dsp/butteraugli.c /home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/aom_dsp/butteraugli.c: In function ‘aom_calc_butteraugli’: /home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/aom_dsp/butteraugli.c:116:3: error: unknown type name ‘JxlPixelFormat’ 116 | JxlPixelFormat pixel_format = { 4, JXL_TYPE_UINT8, JXL_NATIVE_ENDIAN, 0 }; | ^~~~~~~~~~~~~~ /home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/aom_dsp/butteraugli.c:116:38: error: ‘JXL_TYPE_UINT8’ undeclared (first use in this function) 116 | JxlPixelFormat pixel_format = { 4, JXL_TYPE_UINT8, JXL_NATIVE_ENDIAN, 0 }; | ^~~~~~~~~~~~~~ /home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/aom_dsp/butteraugli.c:116:38: note: each undeclared identifier is reported only once for each function it appears in /home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/aom_dsp/butteraugli.c:116:38: warning: excess elements in scalar initializer /home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/aom_dsp/butteraugli.c:116:38: note: (near initialization for ‘pixel_format’) /home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/aom_dsp/butteraugli.c:116:54: error: ‘JXL_NATIVE_ENDIAN’ undeclared (first use in this function) 116 | JxlPixelFormat pixel_format = { 4, JXL_TYPE_UINT8, JXL_NATIVE_ENDIAN, 0 }; | ^~~~~~~~~~~~~~~~~ /home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/aom_dsp/butteraugli.c:116:54: warning: excess elements in scalar initializer /home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/aom_dsp/butteraugli.c:116:54: note: (near initialization for ‘pixel_format’) /home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/aom_dsp/butteraugli.c:116:73: warning: excess elements in scalar initializer 116 | JxlPixelFormat pixel_format = { 4, JXL_TYPE_UINT8, JXL_NATIVE_ENDIAN, 0 }; | ^ /home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/aom_dsp/butteraugli.c:116:73: note: (near initialization for ‘pixel_format’) /home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/aom_dsp/butteraugli.c:118:17: error: request for member ‘data_type’ in something not a structure or union 118 | pixel_format.data_type = JXL_TYPE_UINT16; | ^ /home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/aom_dsp/butteraugli.c:118:30: error: ‘JXL_TYPE_UINT16’ undeclared (first use in this function) 118 | pixel_format.data_type = JXL_TYPE_UINT16; | ^~~~~~~~~~~~~~~ /home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/aom_dsp/butteraugli.c:119:17: error: request for member ‘endianness’ in something not a structure or union 119 | pixel_format.endianness = JXL_BIG_ENDIAN; | ^ /home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/aom_dsp/butteraugli.c:119:31: error: ‘JXL_BIG_ENDIAN’ undeclared (first use in this function); did you mean ‘CONFIG_BIG_ENDIAN’? 119 | pixel_format.endianness = JXL_BIG_ENDIAN; | ^~~~~~~~~~~~~~ | CONFIG_BIG_ENDIAN /home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/aom_dsp/butteraugli.c:121:3: error: unknown type name ‘JxlButteraugliApi’ 121 | JxlButteraugliApi *api = JxlButteraugliApiCreate(NULL); | ^~~~~~~~~~~~~~~~~ /home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/aom_dsp/butteraugli.c:121:28: warning: implicit declaration of function ‘JxlButteraugliApiCreate’ [-Wimplicit-function-declaration] 121 | JxlButteraugliApi *api = JxlButteraugliApiCreate(NULL); | ^~~~~~~~~~~~~~~~~~~~~~~ /home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/aom_dsp/butteraugli.c:121:28: warning: initialization of ‘int *’ from ‘int’ makes pointer from integer without a cast [-Wint-conversion] /home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/aom_dsp/butteraugli.c:123:3: warning: implicit declaration of function ‘JxlButteraugliApiSetParallelRunner’ [-Wimplicit-function-declaration] 123 | JxlButteraugliApiSetParallelRunner(api, JxlThreadParallelRunner, runner); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/aom_dsp/butteraugli.c:124:3: warning: implicit declaration of function ‘JxlButteraugliApiSetHFAsymmetry’ [-Wimplicit-function-declaration] 124 | JxlButteraugliApiSetHFAsymmetry(api, hf_asym_val); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/aom_dsp/butteraugli.c:125:3: warning: implicit declaration of function ‘JxlButteraugliApiSetIntensityTarget’ [-Wimplicit-function-declaration] 125 | JxlButteraugliApiSetIntensityTarget(api, (float)target_intensity); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/aom_dsp/butteraugli.c:127:3: error: unknown type name ‘JxlButteraugliResult’ 127 | JxlButteraugliResult *result = JxlButteraugliCompute( | ^~~~~~~~~~~~~~~~~~~~ /home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/aom_dsp/butteraugli.c:127:34: warning: implicit declaration of function ‘JxlButteraugliCompute’ [-Wimplicit-function-declaration] 127 | JxlButteraugliResult *result = JxlButteraugliCompute( | ^~~~~~~~~~~~~~~~~~~~~ /home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/aom_dsp/butteraugli.c:127:34: warning: initialization of ‘int *’ from ‘int’ makes pointer from integer without a cast [-Wint-conversion] /home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/aom_dsp/butteraugli.c:133:3: warning: implicit declaration of function ‘JxlButteraugliResultGetDistmap’ [-Wimplicit-function-declaration] 133 | JxlButteraugliResultGetDistmap(result, &distmap, &row_stride); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/aom_dsp/butteraugli.c:134:36: warning: implicit declaration of function ‘JxlButteraugliResultGetDistance’ [-Wimplicit-function-declaration] 134 | cpi->butteraugli_info.distance = JxlButteraugliResultGetDistance(result, 3.0f); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/aom_dsp/butteraugli.c:137:5: warning: implicit declaration of function ‘JxlButteraugliApiDestroy’ [-Wimplicit-function-declaration] 137 | JxlButteraugliApiDestroy(api); | ^~~~~~~~~~~~~~~~~~~~~~~~ /home/app/.cache/paru/clone/aom-av1-lavish-git/src/aom-av1-lavish/aom_dsp/butteraugli.c:138:5: warning: implicit declaration of function ‘JxlButteraugliResultDestroy’ [-Wimplicit-function-declaration] 138 | JxlButteraugliResultDestroy(result); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ [98/499] Building C object CMakeFiles/aom_dsp_common_sse2_intrinsics.dir/aom_dsp/x86/aom_convolve_copy_sse2.c.o [99/499] Building C object CMakeFiles/aom_dsp_encoder.dir/aom_dsp/variance.c.o ninja: build stopped: subcommand failed. ==> ERROR: A failure occurred in build(). ```

So, either the entire codebase needs to be updated or it needs to be built using older commit tree of libjxl where the APIs were still present.

[rokibhasansagar]

Everything up until 4e4f49c of libjxl works well with the build. But, I haven't run Any benchmark operations. So, I can't tell if the other APIs are operational or not.

[shssoichiro]

For now I've pinned libjxl-metrics-git to the latest commit that includes butteraugli.h. This is a workaround but should at least unbreak builds.