benchmark_android时，报错source/tnn/optimizer/graph_matcher/ir.cc:750 map::at: key not found. Segmentation error

[wb014]

1. 环境（environment）

• Build OS and Version: Docker
• RunTime OS Version: Android

• RunTime DEVICE: ARM

  2. Github版本

• branch：master

• commit(optional):

  3. 编译方式(compile method) cmake ../../.. \ -DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK/build/cmake/android.toolchain.cmake \ -DCMAKE_BUILD_TYPE=Release \ -DANDROID_ABI="${ABI}" \ -DANDROID_STL=${STL}\ -DANDROID_NATIVE_API_LEVEL=android-14 \ -DANDROID_TOOLCHAIN=clang \ -DTNN_CPU_ENABLE:BOOL=ON \ -DTNN_ARM_ENABLE:BOOL=ON \ -DTNN_ARM82_ENABLE:BOOL=ON \ -DTNN_OPENCL_ENABLE:BOOL=ON \ -DTNN_HUAWEI_NPU_ENABLE:BOOL=${HUAWEI_NPU_ENABLE} \ -DTNN_OPENMP_ENABLE:BOOL=ON \ -DTNN_TEST_ENABLE:BOOL=ON \ -DTNN_BUILD_BENCHMARK_TEST_LIB_ENABLE:BOOL=ON \ -DTNN_BENCHMARK_MODE:BOOL=ON \ -DTNN_PROFILER_ENABLE:BOOL=${PROFILING} \ -DTNN_BUILD_SHARED:BOOL=ON \ -DBUILD_FOR_ANDROID_COMMAND=true make -j4 }

  4. 编译日志(build log)

  -- android-14 is unsupported. Using minimum supported version 21.
  /home/mjw/Projects_zwb/TNN
  /home/mjw/Projects_zwb/TNN
  -- >>>>>>>>>>>>>
  -- TNN BUILD INFO:
  --  System: Android
  --  Processor: aarch64
  --  Cpu:    ON
  --  X86:    OFF
  --  Arm:    ON
  --  Arm82:  ON
  --  Metal:  OFF
  --  OpenCL: ON
  --  CUDA:   OFF
  --  DSP:    OFF
  --  Atlas:  OFF
  --  TensorRT:   OFF
  --  AppleNPU:   OFF
  --  HuaweiNPU:  
  --  RKNPU:  OFF
  --  Jetson Nano:    OFF
  --  OpenVINO:   OFF
  --  OpenMP: ON
  --  TEST:   ON
  --  --Unit Test:    OFF
  --  Quantization:   OFF
  --  ModelCheck: OFF
  --  DEBUG:  
  --  PROFILE:    OFF
  --  BENCHMARK:  ON
  --  BENCHMARK Layer:    OFF
  --  Model Converter:    OFF
  --  ONNX2TNN Converter: OFF
  --  TNN2MEM:    OFF
  --  BENCHMARK Test Lib: ON
  --  Dynamic Range Quantization: OFF
  --  SHARING_MEM_WITH_OPENGL:    
  Generate OpenCL Source done !!! 
  

  5. 详细描述bug 情况 (Describe the bug) benchmark时报错source/tnn/optimizer/graph_matcher/ir.cc:750 map::at: key not found. 模型为用官方docker进行onnx2tnn转换得到

  6. 运行日志（runtime log）

  
  ./benchmark_models.sh -bs -t CPU
  -- android-14 is unsupported. Using minimum supported version 21.
  /home/mjw/Projects_zwb/TNN
  /home/mjw/Projects_zwb/TNN
  -- >>>>>>>>>>>>>
  -- TNN BUILD INFO:
  --  System: Android
  --  Processor: aarch64
  --  Cpu:    ON
  --  X86:    OFF
  --  Arm:    ON
  --  Arm82:  ON
  --  Metal:  OFF
  --  OpenCL: ON
  --  CUDA:   OFF
  --  DSP:    OFF
  --  Atlas:  OFF
  --  TensorRT:   OFF
  --  AppleNPU:   OFF
  --  HuaweiNPU:  
  --  RKNPU:  OFF
  --  Jetson Nano:    OFF
  --  OpenVINO:   OFF
  --  OpenMP: ON
  --  TEST:   ON
  --  --Unit Test:    OFF
  --  Quantization:   OFF
  --  ModelCheck: OFF
  --  DEBUG:  
  --  PROFILE:    OFF
  --  BENCHMARK:  ON
  --  BENCHMARK Layer:    OFF
  --  Model Converter:    OFF
  --  ONNX2TNN Converter: OFF
  --  TNN2MEM:    OFF
  --  BENCHMARK Test Lib: ON
  --  Dynamic Range Quantization: OFF
  --  SHARING_MEM_WITH_OPENGL:    
  Generate OpenCL Source done !!! 

-- Build TNN OpenCL -- Configuring done -- Generating done -- Build files have been written to: /home/mjw/Projects_zwb/TNN/benchmark/benchmark_android/build Scanning dependencies of target TNNArm82 Scanning dependencies of target TNNArm Consolidate compiler generated dependencies of target TNNArm82 Consolidate compiler generated dependencies of target TNNArm Consolidate compiler generated dependencies of target TNNOpenCL [ 6%] Built target TNNArm82 Consolidate compiler generated dependencies of target gflags_nothreads_static Consolidate compiler generated dependencies of target TNNCpu [ 7%] Built target gflags_nothreads_static [ 8%] Building CXX object source/tnn/device/opencl/CMakeFiles/TNNOpenCL.dir/cl/opencl_program.cc.o [ 23%] Built target TNNArm [ 40%] Built target TNNCpu [ 54%] Built target TNNOpenCL Consolidate compiler generated dependencies of target TNN [ 54%] Linking CXX shared library libTNN.so [ 99%] Built target TNN Consolidate compiler generated dependencies of target TNNBenchmarkTest Consolidate compiler generated dependencies of target TNNTest [ 99%] Linking CXX executable TNNTest [ 99%] Linking CXX shared library libTNNBenchmarkTest.so [100%] Built target TNNBenchmarkTest [100%] Built target TNNTest ./libTNN.so: 1 file pushed. 36.1 MB/s (8056256 bytes in 0.213s) ./test/libTNNBenchmarkTest.so: 1 file pushed. 21.3 MB/s (211280 bytes in 0.009s) test/TNNTest: 1 file pushed. 14.7 MB/s (191344 bytes in 0.012s) /home/mjw/Projects_zwb/TNN/benchmark/b...hed. 2.8 MB/s (277330 bytes in 0.093s) 2023-11-22 10:08:10 196: E source/tnn/optimizer/graph_matcher/ir.cc:750 map::at: key not found E/tnn: bool tnn::test::CheckResult(std::string, Status) [File test/test.cc][Line 338] create instance failed: code: 0x6000 msg: map::at: key not found /data/local/tmp/tnn-benchmark/benchmar... pulled. 0.0 MB/s (64 bytes in 0.002s) SM-G9810

benchmark device: ARM

Wed Nov 22 10:08:15 CST 2023

[wb014]

我这里测试出来bug可能位置在multiheadattn上，经过self.linear(value)的linear层后，就会报“map::at: key not found”这个error

[wb014]

后续的scores = torch.matmul(query, key)，同样报错。 E/tnn: virtual Status tnn::MatMulLayerResourceGenerator::GenLayerResource(LayerParam *, LayerResource *, std::vector<Blob > &) [File source/tnn/interpreter/layer_resource_generator.cc][Line 544] [WARNNING] can't infer resource shape from MatMul param in benchmark mode, random generator may not be exactly same with the real resource! 请问有解决办法吗，onnx2tnn用的docker最新镜像，benchmark用的tnn master分支

[wb014]

clean the obsolete tnnproto in server and android before convert and benchmark, the question not emerge again