search

intel / compute-runtime

Intel® Graphics Compute Runtime for oneAPI Level Zero and OpenCL™ Driver
MIT License
1.1k stars 229 forks source link

Some of the tests from neo_shared_tests and igdrcl_tests are failing #662

Closed yktyagi closed 10 months ago

yktyagi commented 12 months ago

Setup / deps Latest release versions of everything, built from source in Yocto

compute-runtime: 23.17.26241.22 intel-gmmlib : 22.3.5 IGC: igc-1.0.13700.14 level-zero: v1.11.0 opencl-clang : 15.0.0 GCC : 13.1

Issue Tests run on the target fail:

neo_shared_test:

======================== == PVC ULTs FAILED ==

Tests run: 7856 Tests passed: 7748 Tests skipped: 82 Tests failed: 26 Tests disabled: 1 Time elapsed: 1016 ms

[ FAILED ][ PVC ][ 0 ] BuiltInSharedTest.GivenValidBuiltinTypeAndAnyTypeWhenGettingBuiltinCodeThenNonEmptyBuiltinIsReturned [ FAILED ][ PVC ][ 0 ] CommandStreamReceiverSystolicTests.givenSystolicModeChangedWhenFlushTaskCalledThenSystolicStateIsUpdated_SystolicSupport [ FAILED ][ PVC ][ 0 ] CompilerInterfaceCachedTests.GivenCachedBinaryWhenBuildingThenSuccessIsReturned [ FAILED ][ PVC ][ 0 ] CompilerInterfaceTest.WhenCompilingToIsaThenSuccessIsReturned [ FAILED ][ PVC ][ 0 ] CompilerInterfaceTest.WhenPreferredIntermediateRepresentationSpecifiedThenPreserveIt [ FAILED ][ PVC ][ 0 ] CompilerInterfaceTest.whenIgcTranslatorReturnsNullptrThenBuildFailsGracefully [ FAILED ][ PVC ][ 0 ] CompilerInterfaceTest.GivenOptionsWhenCompilingToIsaThenSuccessIsReturned [ FAILED ][ PVC ][ 0 ] CompilerInterfaceTest.WhenCompilingToIrThenSuccessIsReturned [ FAILED ][ PVC ][ 0 ] CompilerInterfaceTest.WhenLinkIsCalledThenOclGenBinIsTheTranslationTarget [ FAILED ][ PVC ][ 0 ] CompilerInterfaceTest.WhenCreateLibraryIsCalledThenLlvmBcIsUsedAsIntermediateRepresentation [ FAILED ][ PVC ][ 0 ] CompilerInterfaceTest.givenUpdatedSpecConstValuesWhenBuildProgramThenProperValuesArePassed [ FAILED ][ PVC ][ 0 ] CommandEncodeStatesTest.givenDispatchInterfaceWhenDpasRequiredIsNotDefaultThenPipelineSelectCommandAdded_IsWithinXeGfxFamily [ FAILED ][ PVC ][ 0 ] CommandEncodeStatesTest.givenDebugVariableWhenEncodeStateIsCalledThenSystolicValueIsOverwritten_IsWithinXeGfxFamily [ FAILED ][ PVC ][ 0 ] CommandEncodeStatesTest.givenCommandContainerWithKernelDpasThenSystolicModeEnabled_IsWithinXeGfxFamily [ FAILED ][ PVC ][ 0 ] EngineNodeHelperPvcTests.givenCccsDisabledButDebugVariableSetWhenGetGpgpuEnginesCalledThenSetCccsProperly [ FAILED ][ PVC ][ 0 ] EngineNodeHelperPvcTests.givenCccsDisabledWhenGetGpgpuEnginesCalledThenDontSetCccs [ FAILED ][ PVC ][ 0 ] EngineNodeHelperPvcTests.givenCCSEngineWhenCallingIsCooperativeDispatchSupportedThenTrueIsReturned [ FAILED ][ PVC ][ 0 ] EngineNodeHelperPvcTests.givenCCCSEngineAndRevisionBWhenCallingIsCooperativeDispatchSupportedThenFalseIsReturned [ FAILED ][ PVC ][ 0 ] EngineNodeHelperPvcTests.givenBcsDisabledWhenGetEnginesCalledThenDontCreateAnyBcs [ FAILED ][ PVC ][ 0 ] EngineNodeHelperPvcTests.givenOneBcsEnabledWhenGetEnginesCalledThenCreateOnlyOneBcs [ FAILED ][ PVC ][ 0 ] EngineNodeHelperPvcTests.givenNotAllCopyEnginesWhenSettingEngineTableThenDontAddUnsupported [ FAILED ][ PVC ][ 0 ] EngineNodeHelperPvcTests.givenOneCcsEnabledWhenGetEnginesCalledThenCreateOnlyOneCcs [ FAILED ][ PVC ][ 0 ] EngineNodeHelperPvcTests.givenCccsAsDefaultEngineWhenGetEnginesCalledThenChangeDefaultEngine [ FAILED ][ PVC ][ 0 ] EngineNodeHelperPvcTests.whenGetGpgpuEnginesThenReturnTwoCccsEnginesAndFourCcsEnginesAndEightLinkCopyEngines [ FAILED ][ PVC ][ 0 ] EngineNodeHelperPvcTests.whenGetGpgpuEnginesThenReturnTwoCccsEnginesAndFourCcsEnginesAndLinkCopyEngines [ FAILED ][ PVC ][ 0 ] PvcProductHelper.givenProductHelperWhenAdditionalKernelExecInfoSupportCheckedThenCorrectValueIsReturned

igdrcl_tests:

/usr/src/debug/intel-compute-runtime/23.17.26241.22-r0/opencl/test/unit_test/api/cl_build_program_tests.inl:670: Failure Expected equality of these values: 0 retVal Which is: -45 SIGSEGV in ULT, on: clBuildProgramTest.givenMultiDeviceProgramWithProgramBuiltForSingleDeviceWhenCreatingKernelThenProgramAndKernelDevicesMatchAndSuccessIsReturned SIGABRT in ULT, on: clBuildProgramTest.givenMultiDeviceProgramWithProgramBuiltForSingleDeviceWhenCreatingKernelThenProgramAndKernelDevicesMatchAndSuccessIsReturned Aborted

All tests were passing in previous release 23.13.26032.30 of compute-runtime.

JablonskiMateusz commented 11 months ago

@yktyagi how do you run these tests? could you share command line?

yktyagi commented 11 months ago

We cross compile intel-compute-runtime in yocto build.

We are running the tests on target with following command.

/usr/bin/neo_shared_tests

/usr/bin/igdrcl_tests

These tests were passing in previous release 23.13.26032.30 of compute-runtime.

JablonskiMateusz commented 11 months ago

please use command line used in runners (e.g. run_pvc_3_shared_tests), default revision was changed for pvc, it is included in runner

yktyagi commented 11 months ago

I was not able to locate this runner file. Can you please point me to it's location?

JablonskiMateusz commented 11 months ago

it is not a file, it is a custom build target which runs unit tests for specific product. You may build run_unit_tests to execute all unit tests

yktyagi commented 11 months ago

I extracted the command from runner and executed it on the target.

neo_shared_tests --product pvc --slices 1 --subslices 2 --eu_per_ss 3 --rev_id 4

PVC tests are still failing.

product family: pvc (1271) chdir into shared/pvc/4 directory failed. This might cause test failures. enable SIGALRM handler: 1 set timeout to: 45 enable SIGSEGV handler: 1 enable SIGABRT handler: 1 Iteration: 1 Iteration: 1. random_seed: 0

======================== == PVC ULTs FAILED ==

Tests run: 7856 Tests passed: 7748 Tests skipped: 82 Tests failed: 26 Tests disabled: 1 Time elapsed: 959 ms

[ FAILED ][ PVC ][ 0 ] BuiltInSharedTest.GivenValidBuiltinTypeAndAnyTypeWhenGettingBuiltinCodeThenNonEmptyBuiltinIsReturned [ FAILED ][ PVC ][ 0 ] CommandStreamReceiverSystolicTests.givenSystolicModeChangedWhenFlushTaskCalledThenSystolicStateIsUpdated_SystolicSupport [ FAILED ][ PVC ][ 0 ] CompilerInterfaceCachedTests.GivenCachedBinaryWhenBuildingThenSuccessIsReturned [ FAILED ][ PVC ][ 0 ] CompilerInterfaceTest.WhenCompilingToIsaThenSuccessIsReturned ........

I see the follwing log when the test starts:

"chdir into shared/pvc/4 directory failed. This might cause test failures."

I checked that during build inside build/bin/shared/pvc/ only one directory named "3" is getting created and which is empty. Could this be a reason for test failure?

JablonskiMateusz commented 11 months ago

Could you try with --rev_id 3 ?

yktyagi commented 11 months ago

neo_shared_tests are passing with --rev_id 3

--rev_id 4 [----------] Global test environment tear-down [==========] 7856 tests from 918 test suites ran. (8757 ms total) [ PASSED ] 7748 tests. [ SKIPPED ] 82 tests, listed below: ... 26 FAILED TESTS YOU HAVE 1 DISABLED TEST

--rev_id 3

[----------] Global test environment tear-down [==========] 7856 tests from 918 test suites ran. (8490 ms total) [ PASSED ] 7774 tests. [ SKIPPED ] 82 tests, listed below: .... YOU HAVE 1 DISABLED TEST

JablonskiMateusz commented 11 months ago

This is expected behavior from our side. Our main use case for running unit tests is to build run_unit_tests target.