building unit_tests/test_files/vme_kernels.cl fails

[stef]

when i try to build this fine project for alpine linux i get the following errors:

1:55:15: warning: sampler initializer has invalid Filter Mode bits
    sampler_t vme_samp = 0;
              ^
1:65:48: error: use of undeclared identifier 'NULL'
        if (prediction_motion_vector_buffer != NULL) {
                                               ^
1:76:9: error: use of type 'intel_sub_group_avc_ime_payload_t' requires cl_intel_device_side_avc_motion_estimation extension to be enabled
        intel_sub_group_avc_ime_payload_t payload = intel_sub_group_avc_ime_initialize(srcCoord, partition_mask, sad_adjustment);
        ^
1:76:53: error: use of declaration 'intel_sub_group_avc_ime_initialize' requires cl_intel_device_side_avc_motion_estimation extension to be enabled
        intel_sub_group_avc_ime_payload_t payload = intel_sub_group_avc_ime_initialize(srcCoord, partition_mask, sad_adjustment);
                                                    ^
1:77:74: error: use of undeclared identifier 'CLK_AVC_ME_SEARCH_WINDOW_16x12_RADIUS_INTEL'
        payload = intel_sub_group_avc_ime_set_single_reference(refCoord, CLK_AVC_ME_SEARCH_WINDOW_16x12_RADIUS_INTEL, payload);
                                                                         ^
1:79:9: error: use of type 'intel_sub_group_avc_ime_result_t' requires cl_intel_device_side_avc_motion_estimation extension to be enabled
        intel_sub_group_avc_ime_result_t result = intel_sub_group_avc_ime_evaluate_with_single_reference(srcImg, refImg, vme_samp, payload);
        ^
1:91:13: error: use of type 'intel_sub_group_avc_ref_payload_t' requires cl_intel_device_side_avc_motion_estimation extension to be enabled
            intel_sub_group_avc_ref_payload_t payload = intel_sub_group_avc_fme_initialize(srcCoord, mvs, major_shape, minor_shapes, directions, pixel_mode, sad_adjustment);
            ^
1:91:57: error: use of declaration 'intel_sub_group_avc_fme_initialize' requires cl_intel_device_side_avc_motion_estimation extension to be enabled
            intel_sub_group_avc_ref_payload_t payload = intel_sub_group_avc_fme_initialize(srcCoord, mvs, major_shape, minor_shapes, directions, pixel_mode, sad_adjustment);
                                                        ^
1:92:13: error: use of type 'intel_sub_group_avc_ref_result_t' requires cl_intel_device_side_avc_motion_estimation extension to be enabled
            intel_sub_group_avc_ref_result_t result = intel_sub_group_avc_ref_evaluate_with_single_reference(srcImg, refImg, vme_samp, payload);
            ^
1:109:37: error: use of undeclared identifier 'NULL'
            if (residuals_buffer != NULL) {
                                    ^

Build failed with error code: -11
Command was: /home/s/tasks/aports/ugly/compute-runtime/src/compute-runtime-20.08.15750/build/bin/ocloc -q -file vme_kernels.cl -device glk -64 -out_dir /home/s/tasks/aports/ugly/compute-runtime/src/compute-runtime-20.08.15750/build/bin/Gen9lp/test_files/x64/
make[2]: *** [unit_tests/CMakeFiles/prepare_test_kernels_Gen9lp.dir/build.make:410: bin/Gen9lp/test_files/x64/vme_kernels_Gen9lp.spv] Error 245
make[2]: Leaving directory '/home/s/tasks/aports/ugly/compute-runtime/src/compute-runtime-20.08.15750/build'
make[1]: *** [CMakeFiles/Makefile2:7162: unit_tests/CMakeFiles/prepare_test_kernels_Gen9lp.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

the 1st error seems to be that NULL should be defined, grepping through the sources i found it defined in compute-runtime-20.08.15750/runtime/scheduler/scheduler.cl - at least i could not find it anywhere else (apart from the usual stddef.h, which is not included here). why is NULL not defined here?

the next error that seems to be disabled is cl_intel_device_side_avc_motion_estimation, which seems to be coming from intel-graphics-compiler-igc-1.0.3390/IGC/BiFModule/Languages/OpenCL/PreRelease/opencl_cth_pre_release.h and indeed in src/intel-graphics-compiler-igc-1.0.3390/build/IGC/BiFModule/CMakeFiles/BiFModuleOcl.dir/build.make it seems that -Dcl_intel_device_side_avc_motion_estimation is provided when doing:

src/intel-graphics-compiler-igc-1.0.3390/build/IGC/BiFModule/CMakeFiles/BiFModuleOcl.dir/build.make:2268:
cd /home/s/tasks/aports/ugly/igc/src/intel-graphics-compiler-igc-1.0.3390/build/IGC/BiFModule && /usr/bin/clang-9 -cc1 -x cl -fblocks -fpreserve-vec3-type -opencl-builtins -triple=spir64 -w -emit-llvm-bc -o /home/s/tasks/aports/ugly/igc/src/intel-graphics-compiler-igc-1.0.3390/build/IGC/Release/bif/./IBiF_PreRelease_int.bc_IBIF_PreRelease_Impl__cl__0.bc.tmp -include /home/s/tasks/aports/ugly/igc/src/intel-graphics-compiler-igc-1.0.3390/build/IGC/Release/bif/./opencl_cth.h -I /home/s/tasks/aports/ugly/igc/src/intel-graphics-compiler-igc-1.0.3390/IGC/BiFModule/Languages/OpenCL -I /home/s/tasks/aports/ugly/igc/src/intel-graphics-compiler-igc-1.0.3390/IGC/BiFModule/Languages/OpenCL/PointerSize -I /home/s/tasks/aports/ugly/igc/src/intel-graphics-compiler-igc-1.0.3390/IGC/BiFModule/Headers -I /home/s/tasks/aports/ugly/igc/src/intel-graphics-compiler-igc-1.0.3390/IGC/BiFModule/../AdaptorOCL/ocl_igc_shared/device_enqueue -D__EXECUTION_MODEL_DEBUG=1 -D__OPENCL_C_VERSION__=200 -D__IGC_BUILD__ -Dcl_khr_f16 -Dcl_khrfp64 -Dcl_khr_gl_msaa_sharing -Dcl_khr_mipmap_image -Dcl_khr_depth_images -Dcl_intel_subgroups_short -Dcl_intel_subgroups_char -Dcl_intel_subgroups_long -Dcl_intel_subgroup_local_block_io -Dcl_intel_64bit_global_atomics_placeholder -Dcl_intel_device_side_avc_vme_enable -Dcl_intel_device_side_avc_motion_estimation -D__VME_TYPES_DEFINED__ -finclude-default-header -cl-std=CL2.0 /home/s/tasks/aports/ugly/igc/src/intel-graphics-compiler-igc-1.0.3390/IGC/BiFModule/Languages/OpenCL/PreRelease/IBIF_PreRelease_Impl.cl

this extension is also enabled in src/intel-graphics-compiler-igc-1.0.3390/build/IGC/Release/bif/opencl_cth.h:6962

i'm a bit confused why this extension is not enabled.

thhe last error seems to be related, in src/intel-graphics-compiler-igc-1.0.3390/build/IGC/Release/bif/opencl_cth.h:6754 the following code is to be found:

#if defined(cl_intel_device_side_avc_motion_estimation)
// Device side VME not defined in Clang.
#define CLK_AVC_ME_SEARCH_WINDOW_16x12_RADIUS_INTEL        0x9

which definces the missing symbol, again it seems the motion estimation extension is not enabled or rather in this case the symbol is not defined that i suppose should be if the extension would be enabled.

a simple bruteforce solution is to simply add these at the beginning of vme_kernels.cl

#pragma OPENCL EXTENSION cl_intel_subgroups : enable
#pragma OPENCL EXTENSION cl_intel_device_side_avc_motion_estimation : enable
#ifndef CLK_AVC_ME_SEARCH_WINDOW_16x12_RADIUS_INTEL
#define CLK_AVC_ME_SEARCH_WINDOW_16x12_RADIUS_INTEL        0x9
#endif
#ifndef NULL
#define NULL 0L
#endif

i'm sure there is a much cleaner solution to this though.

[JacekDanecki]

Can you provide information about opencl-clang and spirv-llvm-translator revisions you are using? How did you build them, as a separate components, or as projects in llvm tree? Have you built IGC using llvm/clang sources, or did you use alpine binaries? What parameters have you added to cmake during opencl-clang, spirv-llvm-translator, igc build?

[stef]

sure. sorry for not providing that info earlier! here it goes: clang and llvm are from the distro, both version 9.0.1

opencl-clang version is 9.0.0, cmake parameters like this:

LLVM_CONFIG=/usr/lib/llvm9/bin/llvm-config \
      cmake -Wno-dev -DCMAKE_INSTALL_PREFIX=/usr ..

SPIRV-LLVM-Translator is v 9.0.0-1, cmake was called like this:

CXXFLAGS='-fPIC' CFLAGS='-fPIC' \
  LLVM_CONFIG=/usr/lib/llvm9/bin/llvm-config \
  cmake -Wno-dev -DCMAKE_INSTALL_PREFIX=/usr ..

igc v1.0.3390 has been cmaked like this:

  LLVM_CONFIG=/usr/lib/llvm9/bin/llvm-config \
       CXXFLAGS=-Wno-alloca-larger-than \
       cmake \
      -Wno-dev \
      -DIGC_PREFERRED_LLVM_VERSION=9.0.1 \
      -DIGC_OPTION__ARCHITECTURE_TARGET=Linux64 \
      -DCMAKE_BUILD_TYPE=Release \
      -DCMAKE_INSTALL_PREFIX=/usr \
      ..

everything has been built seperately, out-of-tree. since i intend to package all of this for alpine, and then i would like to avoid to have redundant llvm or clang packages just for opencl. for your convenience i pushed my WIP buildscripts: https://github.com/aports-ugly/aports/commit/7b538c402c4f021ce9478a4dfef1aab2d5362600

[stef]

oops i also pushed some unrelated packages, looking at the respective directories might be easier to look at, than the diffs:

• https://github.com/aports-ugly/aports/tree/7b538c402c4f021ce9478a4dfef1aab2d5362600/ugly/compute-runtime
• https://github.com/aports-ugly/aports/blob/7b538c402c4f021ce9478a4dfef1aab2d5362600/ugly/gmmlib
• https://github.com/aports-ugly/aports/blob/7b538c402c4f021ce9478a4dfef1aab2d5362600/ugly/igc
• https://github.com/aports-ugly/aports/blob/7b538c402c4f021ce9478a4dfef1aab2d5362600/ugly/opencl-clang
• https://github.com/aports-ugly/aports/blob/7b538c402c4f021ce9478a4dfef1aab2d5362600/ugly/spirv-llvm-translator

[JacekDanecki]

As I can see in https://github.com/aports-ugly/aports/blob/7b538c402c4f021ce9478a4dfef1aab2d5362600/ugly/compute-runtime/musl-compat.patch you defined

#define RTLD_DEEPBIND 0

but it's defined as

#define RTLD_DEEPBIND   0x00008 /* Use deep binding.  */

in glibc-headers. Is this flag supported in musl? You can set SANITIZER_BUILD to not use RTLD_DEEPBIND flag.

[stef]

no, musl does not support deepbinding. is it ok to set SANITIZER_BUILD globally via CXXFLAGS or should i only set it for the affected linux_os directory? i have a build running with a global SANITIZER_BUILD flag set, we'll see how that goes.

thank you very much for your support and prompt responses!

[JacekDanecki]

You can set SANITIZER_BUILD using CXXFLAGS globally. I'll prepare setup with alpine to reproduce issue using your build scripts.

[stef]

btw the "makedepends" variable in the buildscripts might be a bit underpopulated, i've not resolved those yet, i'm doing this on my fully installed system, on a pristine system there will probably a bunch of makedepends missing, sorry for that but this is very much work in progress.

[stef]

this seems to be closely related to #265, we might want to merge the two issues.

[AdamCetnerowski]

Closing as duplicate