search

intel / media-driver

Intel Graphics Media Driver to support hardware decode, encode and video processing.
https://github.com/intel/media-driver/wiki
Other
995 stars 346 forks source link

Build fail with -DGEN8=OFF or -DGEN10=OFF #1065

Closed eero-t closed 1 year ago

eero-t commented 4 years ago

When building media driver sources from latest release tag "intel-media-20.3.0": cmake -G Ninja -DGEN8=OFF -DGEN9=ON -DGEN10=OFF -DGEN11=ON -DGEN12=ON . && ninja

The build fails with:

In file included from ../media_driver/agnostic/common/cm/cm_command_buffer.cpp:44:
../media_driver/agnostic/gen12/cm/cm_hal_g12.h:31:10: fatal error: mhw_render_hwcmd_g8_X.h: No such file or directory
   31 | #include "mhw_render_hwcmd_g8_X.h"
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
eero-t commented 4 years ago

If I change cmake to use "-DGEN8=ON", then the build fails to:

../media_driver/agnostic/gen11/codec/hal/codechal_vdenc_vp9_g11.cpp:41:10: fatal error: igcodeckrn_g10.h: No such file or directory
   41 | #include "igcodeckrn_g10.h"
      |          ^~~~~~~~~~~~~~~~~~
compilation terminated.

Aren't you doing automated build-testing before releases to check that the GEN options in CMake still work?

dvrogozh commented 4 years ago

Aren't you doing automated build-testing before releases to check that the GEN options in CMake still work?

No, we don't. These are advanced options used in rare cases. So far there were no requests asking for such build configurations to be permanently supported. We focus on the driver build which targets all platforms supported by the driver. Supported build options are described in the top level readme file: https://github.com/intel/media-driver#build-options.

eero-t commented 4 years ago

Thanks!

If those options are deprecated legacy ones that're going away, that's fine. Some note [1] in CMake file about their status would be nice though.

However, if media driver wants to avoid adding accidental dependencies between GENs, I would suggest adding tests for building with GENs disabled, as that kind of dependency mess can be very hard to clean up afterwards.

It needs only build testing, and these two alternate configs would cover it pretty well, so it wouldn't take too much testing time either:

[1] I'm nowadays doing all my builds with CMake "-LH" option to see from build logs what config options were set. From that output it's easy to see all options and their descriptions.

mattst88 commented 2 years ago

Even if the options are not tested, it makes sense to simply remove the Gen10 code, since Cannonlake GPUs never shipped.

ChromeOS would appreciate code-size reductions, and Gen10 code is at least 3.8 MiB (>10%, 35 MiB -> 31 MiB):

diff --git a/media_driver/cmake/linux/media_gen_flags_linux.cmake b/media_driver/cmake/linux/media_gen_flags_linux.cmake
index ce53e31a..4d3de925 100644
--- a/media_driver/cmake/linux/media_gen_flags_linux.cmake
+++ b/media_driver/cmake/linux/media_gen_flags_linux.cmake
@@ -121,14 +121,6 @@ if(GEN9_CMPV)
     add_definitions(-DIGFX_GEN9_CMPV_SUPPORTED)
 endif()

-if(GEN10)
-    add_definitions(-DIGFX_GEN10_SUPPORTED)
-endif()
-
-if(GEN10_CNL)
-    add_definitions(-DIGFX_GEN10_CNL_SUPPORTED)
-endif()
-
 if(GEN11)
     add_definitions(-DIGFX_GEN11_SUPPORTED)
 endif()

size output:

      text    data     bss     dec     hex filename
  34734574  2127568    9568 36871710  2329e1e ./build/media_driver/iHD_drv_video.so before 
  30824206  1990320    9600 32824126  1f4db3e ./build/media_driver/iHD_drv_video.so after
mattst88 commented 2 years ago

Please reopen. This is still broken for -DGEN10=OFF.

dvrogozh commented 2 years ago

Reopening (got automatically closed after #1302 merge in).

dvrogozh commented 2 years ago

GEN10=OFF build failures:

$ cmake -DGEN8=OFF -DGEN9=ON -DGEN10=OFF -DGEN11=ON -DGEN12=ON ..
In file included from /home/dvrogozh/git/github/media-driver/media_driver/linux/ult/ult_app/gpu_cmd/gpu_cmd_hcp_pipe_buf_addr.cpp:22:0:
/home/dvrogozh/git/github/media-driver/media_driver/linux/ult/ult_app/gpu_cmd/gpu_cmd_hcp_pipe_buf_addr.h:26:10: fatal error: mhw_vdbox_hcp_hwcmd_g10_X.h: No such file or directory
 #include "mhw_vdbox_hcp_hwcmd_g10_X.h"
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
In file included from /home/dvrogozh/git/github/media-driver/media_driver/linux/ult/ult_app/gpu_cmd/gpu_cmd_hcp_ind_obj_base_addr.cpp:22:0:
/home/dvrogozh/git/github/media-driver/media_driver/linux/ult/ult_app/gpu_cmd/gpu_cmd_hcp_ind_obj_base_addr.h:26:10: fatal error: mhw_vdbox_hcp_hwcmd_g10_X.h: No such file or directory
 #include "mhw_vdbox_hcp_hwcmd_g10_X.h"
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
In file included from /home/dvrogozh/git/github/media-driver/media_driver/linux/ult/ult_app/gpu_cmd/gpu_cmd_factory.cpp:26:0:
/home/dvrogozh/git/github/media-driver/media_driver/linux/ult/ult_app/gpu_cmd/gpu_cmd_hcp_ind_obj_base_addr.h:26:10: fatal error: mhw_vdbox_hcp_hwcmd_g10_X.h: No such file or directory
 #include "mhw_vdbox_hcp_hwcmd_g10_X.h"
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
In file included from /home/dvrogozh/git/github/media-driver/media_driver/linux/ult/ult_app/gpu_cmd/gpu_cmd_mfx_bsp_buf_base_addr.cpp:22:0:
/home/dvrogozh/git/github/media-driver/media_driver/linux/ult/ult_app/gpu_cmd/gpu_cmd_mfx_bsp_buf_base_addr.h:28:10: fatal error: mhw_vdbox_mfx_hwcmd_g10_X.h: No such file or directory
 #include "mhw_vdbox_mfx_hwcmd_g10_X.h"
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
In file included from /home/dvrogozh/git/github/media-driver/media_driver/linux/ult/ult_app/gpu_cmd/gpu_cmd_mfx_ind_obj_base_addr.cpp:22:0:
/home/dvrogozh/git/github/media-driver/media_driver/linux/ult/ult_app/gpu_cmd/gpu_cmd_mfx_ind_obj_base_addr.h:28:10: fatal error: mhw_vdbox_mfx_hwcmd_g10_X.h: No such file or directory
 #include "mhw_vdbox_mfx_hwcmd_g10_X.h"
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
In file included from /home/dvrogozh/git/github/media-driver/media_driver/linux/ult/ult_app/gpu_cmd/gpu_cmd_mfx_pipe_buf_addr.cpp:22:0:
/home/dvrogozh/git/github/media-driver/media_driver/linux/ult/ult_app/gpu_cmd/gpu_cmd_mfx_pipe_buf_addr.h:28:10: fatal error: mhw_vdbox_mfx_hwcmd_g10_X.h: No such file or directory
 #include "mhw_vdbox_mfx_hwcmd_g10_X.h"
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[  8%] Built target devult
[  8%] Building CXX object media_driver/CMakeFiles/iHD_drv_video_OBJ.dir/agnostic/gen11/codec/hal/codechal_vdenc_hevc_g11.cpp.o
[  8%] Building CXX object media_driver/CMakeFiles/iHD_drv_video_OBJ.dir/agnostic/gen11/codec/hal/codechal_vdenc_vp9_g11.cpp.o
/home/dvrogozh/git/github/media-driver/media_driver/agnostic/gen11/codec/hal/codechal_vdenc_hevc_g11.cpp:27:10: fatal error: codechal_vdenc_hevc_g10.h: No such file or directory
 #include "codechal_vdenc_hevc_g10.h"
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
/home/dvrogozh/git/github/media-driver/media_driver/agnostic/gen11/codec/hal/codechal_vdenc_vp9_g11.cpp:41:10: fatal error: igcodeckrn_g10.h: No such file or directory
 #include "igcodeckrn_g10.h"
          ^~~~~~~~~~~~~~~~~~
compilation terminated.

@XinfengZhang : do you remember whether g10 headers usage for g11 path was by intent?

dvrogozh commented 2 years ago

Looking into the code, codechal_vdenc_vp9_g11.cpp is trivial - we can freely remove #include "igcodeckrn_g10.h", nothing depends on 2 definitions it exposes.

codechal_vdenc_hevc_g11.cpp is the problem however. It heavily uses definitions from codechal_vdenc_hevc_g10.h which results in a bunch of errors:

$ less media_driver/agnostic/gen11/codec/hal/codechal_vdenc_hevc_g11.cpp | grep _G10
    PCODECHAL_VDENC_HEVC_STREAMIN_STATE_G10 data = (PCODECHAL_VDENC_HEVC_STREAMIN_STATE_G10)streaminData;
    uint32_t frameWidthInLcus = CODECHAL_GET_WIDTH_IN_BLOCKS(m_frameWidth, CODECHAL_HEVC_MAX_LCU_SIZE_G10);
    uint32_t frameHeightInLcus = CODECHAL_GET_WIDTH_IN_BLOCKS(m_frameHeight, CODECHAL_HEVC_MAX_LCU_SIZE_G10);
        kernelParams->iBTCount = CODECHAL_VDENC_HME_END_G10 - CODECHAL_VDENC_HME_BEGIN_G10;
        kernelParams->iCurbeLength = MOS_ALIGN_CEIL(sizeof(MEDIA_OBJECT_HEVC_VP9_VDENC_ME_CURBE_G10), (size_t)curbeAlignment);
            bindingTable->dwNumBindingTableEntries = CODECHAL_VDENC_HME_END_G10 - CODECHAL_VDENC_HME_BEGIN_G10;
            bindingTable->dwBindingTableStartOffset = CODECHAL_VDENC_HME_BEGIN_G10;
    CODECHAL_VDENC_HEVC_ME_CURBE_G10 curbe;
        sizeof(CODECHAL_VDENC_HEVC_ME_CURBE_G10),
        sizeof(CODECHAL_VDENC_HEVC_ME_CURBE_G10)));
    curbe.DW40._4xMeMvOutputDataSurfIndex = CODECHAL_VDENC_HME_MV_DATA_SURFACE_CM_G10;
    curbe.DW41._16xOr32xMeMvInputDataSurfIndex = (hmeLevel == HME_LEVEL_32x) ? CODECHAL_VDENC_32xME_MV_DATA_SURFACE_CM_G10 : CODECHAL_VDENC_16xME_MV_DATA_SURFACE_CM_G10;
    curbe.DW42._4xMeOutputDistSurfIndex = CODECHAL_VDENC_HME_DISTORTION_SURFACE_CM_G10;
    curbe.DW43._4xMeOutputBrcDistSurfIndex = CODECHAL_VDENC_HME_BRC_DISTORTION_CM_G10;
    curbe.DW44.VMEFwdInterPredictionSurfIndex = CODECHAL_VDENC_HME_CURR_FOR_FWD_REF_CM_G10;
    curbe.DW45.VMEBwdInterPredictionSurfIndex = CODECHAL_VDENC_HME_CURR_FOR_BWD_REF_CM_G10;
    curbe.DW46.VDEncStreamInOutputSurfIndex = CODECHAL_VDENC_HME_VDENC_STREAMIN_OUTPUT_CM_G10;
    curbe.DW47.VDEncStreamInInputSurfIndex = CODECHAL_VDENC_HME_VDENC_STREAMIN_INPUT_CM_G10;
    surfaceCodecParams.dwBindingTableOffset = bindingTable->dwBindingTableEntries[CODECHAL_VDENC_HME_MV_DATA_SURFACE_CM_G10];
        surfaceCodecParams.dwBindingTableOffset = bindingTable->dwBindingTableEntries[CODECHAL_VDENC_32xME_MV_DATA_SURFACE_CM_G10];
        surfaceCodecParams.dwBindingTableOffset = bindingTable->dwBindingTableEntries[CODECHAL_VDENC_16xME_MV_DATA_SURFACE_CM_G10];
        surfaceCodecParams.dwBindingTableOffset = bindingTable->dwBindingTableEntries[CODECHAL_VDENC_HME_DISTORTION_SURFACE_CM_G10];
                surfaceCodecParams.dwBindingTableOffset = bindingTable->dwBindingTableEntries[CODECHAL_VDENC_HME_CURR_FOR_FWD_REF_CM_G10];
            surfaceCodecParams.dwBindingTableOffset = bindingTable->dwBindingTableEntries[CODECHAL_VDENC_HME_FWD_REF_IDX0_CM_G10 + (refIdx * 2)];
                surfaceCodecParams.dwBindingTableOffset = bindingTable->dwBindingTableEntries[CODECHAL_VDENC_HME_CURR_FOR_BWD_REF_CM_G10];
            surfaceCodecParams.dwBindingTableOffset = bindingTable->dwBindingTableEntries[CODECHAL_VDENC_HME_BWD_REF_IDX0_CM_G10 + (refIdx * 2)];
            surfaceCodecParams.dwBindingTableOffset = bindingTable->dwBindingTableEntries[CODECHAL_VDENC_HME_VDENC_STREAMIN_INPUT_CM_G10];
        surfaceCodecParams.dwBindingTableOffset = bindingTable->dwBindingTableEntries[CODECHAL_VDENC_HME_VDENC_STREAMIN_OUTPUT_CM_G10];
cqian2 commented 2 years ago

Hi, Is the issue still reproducible on your side? I used the latest config w/ "-DGEN10=OFF", but did not encounter any issue here. Please find below main log.

~/build_media# rm -rf ~/build_media# cmake -DCMAKE_INSTALL_PREFIX=/usr -DGEN10=OFF ../media-driver -- The C compiler identification is GNU 9.4.0 -- The CXX compiler identification is GNU 9.4.0 -- Check for working C compiler: /usr/bin/cc -- Check for working C compiler: /usr/bin/cc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /usr/bin/c++ -- Check for working CXX compiler: /usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- CMAKE_INSTALL_PREFIX = /usr -- Found PkgConfig: /usr/bin/pkg-config (found version "0.29.1") -- Checking for module 'libva>=1.8.0' -- Found libva, version 1.17.0 BUILD_TYPE not defined, default to: release -- Found X11: /usr/include -- Looking for XOpenDisplay in /usr/lib/x86_64-linux-gnu/libX11.so;/usr/lib/x86_64-linux-gnu/libXext.so -- Looking for XOpenDisplay in /usr/lib/x86_64-linux-gnu/libX11.so;/usr/lib/x86_64-linux-gnu/libXext.so - found -- Looking for gethostbyname -- Looking for gethostbyname - found -- Looking for connect -- Looking for connect - found -- Looking for remove -- Looking for remove - found -- Looking for shmat -- Looking for shmat - found -- Looking for IceConnectionNumber in ICE -- Looking for IceConnectionNumber in ICE - found -- Checking for module 'igdgmm>=12.0.0' -- Found igdgmm, version 12.2.0 -- media -- PLATFORM = linux -- media -- ARCH = 64 -- media -- CMAKE_CURRENT_LIST_DIR = /home/intel/Desktop/dev_media/gfx-driver/Source/media-driver/media_driver -- media -- INCLUDED_LIBS = -- media -- LIB_NAME = iHD_drv_video -- media -- OUTPUT_NAME = -- media -- BUILD_TYPE/UFO_BUILD_TYPE/CMAKE_BUILD_TYPE = release/release/Release -- media -- LIBVA_INSTALL_PATH = -- media -- MEDIA_VERSION = 22.6.0 -- media -- X11 Found -- Checking for module 'libva-x11' -- Found libva-x11, version 1.16.0 -- /home/intel/Desktop/dev_media/gfx-driver/Source/media-driver/media_driver/agnostic/common/vp/kernel/media_srcs.cmake doesn't exist, macro(media_include_subdirectory) just does nothing -- /home/intel/Desktop/dev_media/gfx-driver/Source/media-driver/media_softlet/agnostic/common/media_interfaces/mmd/media_srcs.cmake doesn't exist, macro(media_include_subdirectory) just does nothing -- LIBVA_DRIVERS_PATH = /usr/lib/x86_64-linux-gnu/dri -- Configuring done -- Generating done -- Build files have been written to: /home/intel/Desktop/dev_media/gfx-driver/Source/build_media ~/build_media# make -j8 Scanning dependencies of target igfxcmrt Scanning dependencies of target iHD_drv_video_SSE2 Scanning dependencies of target iHD_drv_video_CP Scanning dependencies of target iHD_drv_video_SSE4 Scanning dependencies of target iHD_drv_video_mos Scanning dependencies of target iHD_drv_video_VP Scanning dependencies of target iHD_drv_video_COMMON Scanning dependencies of target iHD_drv_video_CODEC [ 0%] Building CXX object cmrtlib/linux/CMakeFiles/igfxcmrt.dir//agnostic/share/cm_printf_host.cpp.o [ 0%] Building CXX object media_driver/CMakeFiles/iHD_drv_video_SSE4.dir/linux/common/cm/hal/osservice/cm_mem_os_sse4_impl.cpp.o [ 0%] Building CXX object media_driver/CMakeFiles/iHD_drv_video_SSE2.dir/agnostic/common/cm/cm_mem_sse2_impl.cpp.o [ 0%] Building CXX object media_driver/CMakeFiles/iHD_drv_video_CP.dir/__/media_softlet/agnostic/common/cp/encodecp.cpp.o [ 0%] Building CXX object cmrtlib/linux/CMakeFiles/igfxcmrt.dir//agnostic/share/cm_rt_helpers.cpp.o [ 0%] Building CXX object cmrtlib/linux/CMakeFiles/igfxcmrt.dir//agnostic/share/cm_avs_state_msg_ex.cpp.o [ 0%] Building CXX object cmrtlib/linux/CMakeFiles/igfxcmrt.dir//agnostic/hardware/cm_device.cpp.o [ 1%] Building CXX object cmrtlib/linux/CMakeFiles/igfxcmrt.dir//agnostic/hardware/cm_device_export.cpp.o [ 1%] Building CXX object cmrtlib/linux/CMakeFiles/igfxcmrt.dir//agnostic/hardware/cm_surface_manager.cpp.o [ 1%] Building CXX object cmrtlib/linux/CMakeFiles/igfxcmrt.dir//agnostic/hardware/cm ... ... ... [ 99%] Building CXX object media_driver/CMakeFiles/iHD_drv_video_CODEC.dir//media_softlet/agnostic/common/codec/hal/enc/shared/features/encode_lpla.cpp.o [100%] Building CXX object media_driver/CMakeFiles/iHD_drv_video_CODEC.dir//media_softlet/agnostic/common/codec/hal/enc/shared/bufferMgr/encode_recycle_res_queue.cpp.o [100%] Building CXX object media_driver/CMakeFiles/iHD_drv_video_CODEC.dir//media_softlet/agnostic/common/codec/hal/enc/shared/bufferMgr/encode_recycle_resource.cpp.o [100%] Building CXX object media_driver/CMakeFiles/iHD_drv_video_CODEC.dir//media_softlet/agnostic/common/codec/hal/enc/shared/bufferMgr/encode_tracked_buffer.cpp.o [100%] Building CXX object media_driver/CMakeFiles/iHD_drv_video_CODEC.dir//media_softlet/agnostic/common/codec/hal/enc/shared/bufferMgr/encode_tracked_buffer_queue.cpp.o [100%] Building CXX object media_driver/CMakeFiles/iHD_drv_video_CODEC.dir//media_softlet/agnostic/common/codec/hal/enc/shared/bufferMgr/encode_tracked_buffer_slot.cpp.o [100%] Building CXX object media_driver/CMakeFiles/iHD_drv_video_CODEC.dir//media_softlet/agnostic/common/codec/hal/enc/shared/bufferMgr/encode_allocator.cpp.o [100%] Building CXX object media_driver/CMakeFiles/iHD_drv_video_CODEC.dir//media_softlet/agnostic/common/codec/hal/enc/shared/scalability/encode_scalability_multipipe.cpp.o [100%] Building CXX object media_driver/CMakeFiles/iHD_drv_video_CODEC.dir//media_softlet/agnostic/common/codec/hal/enc/shared/scalability/encode_scalability_option.cpp.o [100%] Building CXX object media_driver/CMakeFiles/iHD_drv_video_CODEC.dir//media_softlet/agnostic/common/codec/hal/enc/shared/scalability/encode_scalability_singlepipe.cpp.o [100%] Building CXX object media_driver/CMakeFiles/iHD_drv_video_CODEC.dir/__/media_softlet/agnostic/common/codec/hal/enc/shared/statusreport/encode_status_report.cpp.o [100%] Building CXX object media_driver/CMakeFiles/iHD_drv_video_CODEC.dir//media_softlet/agnostic/common/codec/hal/enc/shared/mmc/encode_mem_compression.cpp.o [100%] Building CXX object media_driver/CMakeFiles/iHD_drv_video_CODEC.dir/__/media_softlet/agnostic/common/codec/hal/shared/codec_hw_next.cpp.o [100%] Built target iHD_drv_video_CODEC Scanning dependencies of target iHD_drv_video_STATIC Scanning dependencies of target iHD_drv_video [100%] Linking CXX shared library iHD_drv_video.so [100%] Linking CXX static library iHD_drv_video.a [100%] Built target iHD_drv_video [100%] Built target iHD_drv_video_STATIC ~/build_media#

mattst88 commented 2 years ago

Yes, it's reproduceable with

$ cmake -DGEN10=OFF -DENABLE_REQUIRED_GEN_CODE=OFF -GNinja -Bbuild
$ ninja -C build

The key is that you have to turn off ENABLE_REQUIRED_GEN_CODE which I added in commit 0e75e91373d5a0d7ae0775743fdc5bc4ad9e7842 in order to disable the media kernels.

mattst88 commented 2 years ago

@cqian2: are you able to reproduce the issue?

cqian2 commented 2 years ago

yes, we could reproduce the issue now. We will give solution in next two weeks.

cqian2 commented 1 year ago

It seems to be sharing the root cause w/ issue - https://github.com/intel/media-driver/issues/1540 @Jexu may help fix both.

mattst88 commented 1 year ago

Any progress?