search

ARM-software / armnn

Arm NN ML Software. The code here is a read-only mirror of https://review.mlplatform.org/admin/repos/ml/armnn
https://developer.arm.com/products/processors/machine-learning/arm-nn
MIT License
1.14k stars 307 forks source link

Unitests failed #771

Closed wxmwy closed 1 month ago

wxmwy commented 1 month ago

I compiled the armnn on the yitian710 machine (support armv9). The build was successful, but the unit test has wrong. Any suggestions? Thanks.

build env:

flatbuffers v23.5.26
protobuf v3.12.0
armnn v24.02
ComputeLibrary v24.02
boost v1.64.0
tensorflow v2.15.0
onnx 553df22c

test logs:

[doctest] doctest version is "2.4.6"
[doctest] run with "--help" for options
Warning: Timed out waiting on profiling service activation for 3000.07 ms
Warning: Timed out waiting on profiling service activation for 3000.07 ms
Warning: Timed out waiting on profiling service activation for 3000.07 ms
Warning: Timed out waiting on profiling service activation for 3000.07 ms
Warning: Timed out waiting on profiling service activation for 3000.07 ms
===============================================================================
/root/wy/armnn-dist/armnn/src/backends/neon/test/NeonEndToEndTests.cpp:191:
TEST SUITE: NeonEndToEnd
TEST CASE:  NeonPowerEndToEndTestFloat32

/root/wy/armnn-dist/armnn/src/backends/backendsCommon/test/EndToEndTestImpl.hpp:180: ERROR: CHECK( Compare<ArmnnOType>(it.second[i], out[i], tolerance) == true ) is NOT correct!
  values: CHECK( false == true )
  logged: Actual output: 0. Expected output:1

/root/wy/armnn-dist/armnn/src/backends/backendsCommon/test/EndToEndTestImpl.hpp:180: ERROR: CHECK( Compare<ArmnnOType>(it.second[i], out[i], tolerance) == true ) is NOT correct!
  values: CHECK( false == true )
  logged: Actual output: 25. Expected output:25

/root/wy/armnn-dist/armnn/src/backends/backendsCommon/test/EndToEndTestImpl.hpp:180: ERROR: CHECK( Compare<ArmnnOType>(it.second[i], out[i], tolerance) == true ) is NOT correct!
  values: CHECK( false == true )
  logged: Actual output: 0. Expected output:25

/root/wy/armnn-dist/armnn/src/backends/backendsCommon/test/EndToEndTestImpl.hpp:180: ERROR: CHECK( Compare<ArmnnOType>(it.second[i], out[i], tolerance) == true ) is NOT correct!
  values: CHECK( false == true )
  logged: Actual output: 25. Expected output:25

/root/wy/armnn-dist/armnn/src/backends/backendsCommon/test/EndToEndTestImpl.hpp:180: ERROR: CHECK( Compare<ArmnnOType>(it.second[i], out[i], tolerance) == true ) is NOT correct!
  values: CHECK( false == true )
  logged: Actual output: 25. Expected output:25

/root/wy/armnn-dist/armnn/src/backends/backendsCommon/test/EndToEndTestImpl.hpp:180: ERROR: CHECK( Compare<ArmnnOType>(it.second[i], out[i], tolerance) == true ) is NOT correct!
  values: CHECK( false == true )
  logged: Actual output: 0. Expected output:9

/root/wy/armnn-dist/armnn/src/backends/backendsCommon/test/EndToEndTestImpl.hpp:180: ERROR: CHECK( Compare<ArmnnOType>(it.second[i], out[i], tolerance) == true ) is NOT correct!
  values: CHECK( false == true )
  logged: Actual output: 9. Expected output:9

/root/wy/armnn-dist/armnn/src/backends/backendsCommon/test/EndToEndTestImpl.hpp:180: ERROR: CHECK( Compare<ArmnnOType>(it.second[i], out[i], tolerance) == true ) is NOT correct!
  values: CHECK( false == true )
  logged: Actual output: 9. Expected output:9

/root/wy/armnn-dist/armnn/src/backends/backendsCommon/test/EndToEndTestImpl.hpp:180: ERROR: CHECK( Compare<ArmnnOType>(it.second[i], out[i], tolerance) == true ) is NOT correct!
  values: CHECK( false == true )
  logged: Actual output: 9. Expected output:9

/root/wy/armnn-dist/armnn/src/backends/backendsCommon/test/EndToEndTestImpl.hpp:180: ERROR: CHECK( Compare<ArmnnOType>(it.second[i], out[i], tolerance) == true ) is NOT correct!
  values: CHECK( false == true )
  logged: Actual output: 0. Expected output:16

===============================================================================
/root/wy/armnn-dist/armnn/src/backends/neon/test/NeonLayerTests.cpp:1646:
TEST SUITE: Compute_ArmComputeNeon
TEST CASE:  RsqrtZero

/root/wy/armnn-dist/armnn/src/armnnTestUtils/UnitTests.hpp:55: ERROR: CHECK( result.m_Result ) is NOT correct!
  values: CHECK( false )
  logged: 2 different values at: [0,0] (nan != inf), [0,1] (nan != -inf)

===============================================================================
/root/wy/armnn-dist/armnn/src/backends/neon/test/NeonLayerTests.cpp:1692:
TEST SUITE: Compute_ArmComputeNeon
TEST CASE:  LogZero

/root/wy/armnn-dist/armnn/src/armnnTestUtils/UnitTests.hpp:55: ERROR: CHECK( result.m_Result ) is NOT correct!
  values: CHECK( false )
  logged: 2 different values at: [0,0] (-88.0297 != -inf), [0,1] (-88.0297 != -inf)

===============================================================================
/root/wy/armnn-dist/armnn/src/backends/neon/test/NeonLayerTests.cpp:1693:
TEST SUITE: Compute_ArmComputeNeon
TEST CASE:  LogNegative

/root/wy/armnn-dist/armnn/src/armnnTestUtils/UnitTests.hpp:55: ERROR: CHECK( result.m_Result ) is NOT correct!
  values: CHECK( false )
  logged: 2 different values at: [0,0] (-175.671 != -nan), [0,1] (-175.688 != -nan)

Warning: Timed out waiting on profiling service activation for 3000.07 ms
Warning: Timed out waiting on profiling service activation for 3000.07 ms
Warning: Timed out waiting on profiling service activation for 3000.07 ms
Warning: Timed out waiting on profiling service activation for 3000.07 ms
===============================================================================
[doctest] test cases:   4300 |   4296 passed |  4 failed | 3 skipped
[doctest] assertions: 802789 | 802776 passed | 13 failed |
[doctest] Status: FAILURE!
Colm-in-Arm commented 1 month ago

Hello wxmwy,

Obviously I don't have access to a yitian710 but I'll ask in Arm Compute Library is they've seen any problems with BinaryOperation::Power, UnaryOperation::Rsqrt or armnn::UnaryOperation::Log operators on their v9 devices.

Colm.

Colm-in-Arm commented 1 month ago

Hello wxmwy,

Have you run the ACL validation tests? This will eliminate Arm NN as a source of the problem. You can build it by adding validation_tests=1 to the scons build. This will create test program "arm_compute_validation".

Could you also tell me the full command line you're using to build ACL please.

Colm.

wxmwy commented 1 month ago

Hello colm,

build with:

scons arch=arm64-v8.2-a-sve extra_cxx_flags="-fPIC" benchmark_tests=1 validation_tests=1 neon=1

run arm_compute_validation got:

Running [1021] 'CPP/TopKV/Validate@PredictionsInfo=Shape=10,20,DataLayout=NCHW,DataType=F16:TargetsInfo=Shape=10,DataLayout=NCHW,DataType=U32:OutputInfo=Shape=10,DataLayout=NCHW,DataType=U8:k=1:Expected=false'
  Wall clock/Wall clock time:    AVG=5.0000 us
Running [1022] 'CPP/TopKV/Validate@PredictionsInfo=Shape=20,10,DataLayout=NCHW,DataType=S8:TargetsInfo=Shape=10,DataLayout=NCHW,DataType=U32:OutputInfo=Shape=10,DataLayout=NCHW,DataType=U8:k=2:Expected=false'
  Wall clock/Wall clock time:    AVG=3.0000 us
Running [1023] 'CPP/TopKV/Validate@PredictionsInfo=Shape=10,10,10,DataLayout=NCHW,DataType=F32:TargetsInfo=Shape=10,DataLayout=NCHW,DataType=U32:OutputInfo=Shape=10,DataLayout=NCHW,DataType=U8:k=3:Expected=false'
  Wall clock/Wall clock time:    AVG=1.0000 us
Running [1024] 'CPP/TopKV/Validate@PredictionsInfo=Shape=20,10,DataLayout=NCHW,DataType=F32:TargetsInfo=Shape=10,DataLayout=NCHW,DataType=U32:OutputInfo=Shape=1,DataLayout=NCHW,DataType=U8:k=4:Expected=false'
  Wall clock/Wall clock time:    AVG=1.0000 us
Running [1025] 'CPP/TopKV/Float'
  Wall clock/Wall clock time:    AVG=31.0000 us
Running [1026] 'CPP/TopKV/QASYMM8'
  Wall clock/Wall clock time:    AVG=11.0000 us
Running [1027] 'CPP/TopKV/QASYMM8_SIGNED'
  Wall clock/Wall clock time:    AVG=10.0000 us
Running [1028] 'CL/UNIT/CompileContext/CompileContextCache'
ERROR: ERROR: clGetDeviceInfo
CONTEXT:
Error code: -5
Running [1029] 'CL/UNIT/DynamicTensor/DynamicTensorType3Single@Level0Shape=12,11,3:Level1Shape=67,31,15'
ERROR: in generate_build_options src/core/CL/CLCompileContext.cpp:284: Non uniform workgroup size is not supported!!
Running [1030] 'CL/UNIT/DynamicTensor/DynamicTensorType3Single@Level0Shape=256,8,12:Level1Shape=11,2,3'
ERROR: in generate_build_options src/core/CL/CLCompileContext.cpp:284: Non uniform workgroup size is not supported!!
Running [1031] 'CL/UNIT/DynamicTensor/DynamicTensorType3Complex@InputShape=[12,12,16, 64,64,16]:WeightsManager=3,3,16,5:BiasShape=5:OutputShape=[12,12,5, 64,64,5]:PadStrideInfo=1,1;1,1,1,1'
ERROR: ERROR: clGetDeviceInfo
CONTEXT:
Error code: -5
Running [1032] 'CL/UNIT/DynamicTensor/DynamicTensorType2Pipeline@InputShape=[12,12,6, 128,128,6]'
ERROR: in generate_build_options src/core/CL/CLCompileContext.cpp:284: Non uniform workgroup size is not supported!!
Running [1033] 'CL/UNIT/GetClContext'
  Wall clock/Wall clock time:    AVG=1.0000 us
Running [1034] 'CL/UNIT/MLGOHeuristics/CorrectDotMLGOShouldLoadCorrectly'
  Wall clock/Wall clock time:    AVG=212.0000 us
Running [1035] 'CL/UNIT/MLGOHeuristics/InvalidDotmlgoSyntaxShouldReturnInvalidStatus'
  Wall clock/Wall clock time:    AVG=5.0000 us
Running [1036] 'CL/UNIT/MLGOHeuristics/InvalidDotmlgoSemanticsShouldReturnInvalidStatus/MismatchesBetweenHeuristicsTableEntriesAndHeuristicTrees'
  Wall clock/Wall clock time:    AVG=29.0000 us
Running [1037] 'CL/UNIT/MLGOHeuristics/InvalidDotmlgoSemanticsShouldReturnInvalidStatus/RepeatedHeuristicsTableEntriesId'
  Wall clock/Wall clock time:    AVG=13.0000 us
Running [1038] 'CL/UNIT/MLGOHeuristics/InvalidDotmlgoSemanticsShouldReturnInvalidStatus/RepeatedHeuristicsTableEntriesIndex'
  Wall clock/Wall clock time:    AVG=13.0000 us
Running [1039] 'CL/UNIT/MLGOHeuristics/InvalidDotmlgoSemanticsShouldReturnInvalidStatus/RepeatedHeuristicTreesId'
  Wall clock/Wall clock time:    AVG=28.0000 us
Running [1040] 'CL/UNIT/MLGOHeuristics/InvalidDotmlgoSemanticsShouldReturnInvalidStatus/EmptyTree'
  Wall clock/Wall clock time:    AVG=11.0000 us
Running [1041] 'CL/UNIT/MLGOHeuristics/InvalidDotmlgoSemanticsShouldReturnInvalidStatus/InvalidTreeMissingRoot'
  Wall clock/Wall clock time:    AVG=27.0000 us
Running [1042] 'CL/UNIT/MLGOHeuristics/InvalidDotmlgoSemanticsShouldReturnInvalidStatus/InvalidTreeMissingNodes'
  Wall clock/Wall clock time:    AVG=20.0000 us
Running [1043] 'CL/UNIT/MLGOHeuristics/InvalidDotmlgoSemanticsShouldReturnInvalidStatus/InvalidTreeRepeatedNodeIds'
  Wall clock/Wall clock time:    AVG=23.0000 us
Running [1044] 'CL/UNIT/MLGOHeuristics/InvalidDotmlgoSemanticsShouldReturnInvalidStatus/InvalidTreeDisjointNodes'
  Wall clock/Wall clock time:    AVG=94.0000 us
Running [1045] 'CL/UNIT/MLGOHeuristics/InvalidDotmlgoSemanticsShouldReturnInvalidStatus/InvalidTreeLoop'
  Wall clock/Wall clock time:    AVG=20.0000 us
Running [1046] 'CL/UNIT/MLGOHeuristics/InvalidDotmlgoSemanticsShouldReturnInvalidStatus/InvalidTreeCycle'
  Wall clock/Wall clock time:    AVG=35.0000 us
Running [1047] 'CL/UNIT/MLGOHeuristics/InvalidDotmlgoSemanticsShouldReturnInvalidStatus/InvalidTreeInvalidFeatures'
  Wall clock/Wall clock time:    AVG=14.0000 us
Running [1048] 'CL/UNIT/MLGOHeuristics/InvalidUsageOfHeuristicsShouldReturnInvalidStatus'
  Wall clock/Wall clock time:    AVG=28.0000 us
Running [1049] 'CL/UNIT/MemoryManager/BlobMemoryManagerSimpleWithinFunctionLevel'
ERROR: in generate_build_options src/core/CL/CLCompileContext.cpp:284: Non uniform workgroup size is not supported!!
Running [1050] 'CL/UNIT/RuntimeContext/MultipleThreadedScheduller'
terminate called after throwing an instance of 'std::runtime_error'
  what():  in generate_build_options src/core/CL/CLCompileContext.cpp:284: Non uniform workgroup size is not supported!!
Aborted (core dumped)

I decided to go straight to the precompiled version. Seems to be working fine. Thanks.