ARM initializing model error: StrideSliceLayer param got wrong size: input dims size: 2

iamjoseph331 commented 1 year ago

1. 环境（environment）

Build OS and Version: Arm64-v8a, armeabi-v7a
RunTime OS Version: Android (Android Studio Emulator)
RunTime DEVICE: ARM

2. Github版本
branch：Master

commit(optional):

3. 编译方式(compile method) CMake完整编译参数(full cmake arguments)

cmake ${TNN_ROOT_PATH} \
  -DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK/build/cmake/android.toolchain.cmake \
  -DDEBUG:BOOL=$DEBUG \
  -DANDROID_ABI="${ABIA64}" \
  -DANDROID_STL=${STL} \
  -DANDROID_NATIVE_API_LEVEL=${ANDROID_API_LEVEL}  \
  -DANDROID_TOOLCHAIN=clang \
  -DBUILD_FOR_ANDROID_COMMAND=true \
  -DTNN_CPU_ENABLE:BOOL=ON \
  -DTNN_ARM_ENABLE:BOOL=$ARM \
  -DTNN_ARM82_ENABLE:BOOL=$ARM82 \
  -DTNN_HUAWEI_NPU_ENABLE:BOOL=$HUAWEI_NPU \
  -DTNN_OPENCL_ENABLE:BOOL=$OPENCL \
  -DTNN_TEST_ENABLE:BOOL=ON \
  -DTNN_BENCHMARK_MODE:BOOL=$BENMARK_MODE \
  -DTNN_OPENMP_ENABLE:BOOL=$OPENMP \
  -DSHARING_MEM_WITH_OPENGL=${SHARING_MEM_WITH_OPENGL} \
  -DTNN_BUILD_SHARED:BOOL=$SHARED_LIB \
  -DPYTHON_EXECUTABLE=/usr/bin/python3
make -j8

4. 编译日志(build log) 将日志粘贴在这里 pastebin

**** step 1: update version.h **** /Users/joseph/Projects/Mobile/TNN/scripts/version rm: /Users/joseph/Projects/Mobile/TNN/scripts/version/version.h: No such file or directory rm: -f: No such file or directory Target: tnn Commit Branch: master Commit Date: 2022-12-21 Commit Hash: 35690301 rm: -f: No such file or directory

**** step 2: start build rpn arm32 **** remove build32 armeabi-v7a -- android-14 is unsupported. Using minimum supported version 16. -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /Users/joseph/Downloads/android-ndk-r21b/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /Users/joseph/Downloads/android-ndk-r21b/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- The ASM compiler identification is Clang with GNU-like command-line -- Found assembler: /Users/joseph/Downloads/android-ndk-r21b/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang /Users/joseph/Projects/Mobile/TNN /Users/joseph/Projects/Mobile/TNN -- Found OpenMP_C: -fopenmp=libomp (found version "3.1") -- Found OpenMP_CXX: -fopenmp=libomp (found version "3.1") -- Found OpenMP: TRUE (found version "3.1")
-- >>>>>>>>>>>>> -- TNN BUILD INFO: -- System: Android -- Processor: armv7-a -- Cpu: ON -- X86: OFF -- Arm: ON -- Arm82: ON -- Metal: OFF -- OpenCL: ON -- CUDA: OFF -- DSP: OFF -- Atlas: OFF -- TensorRT: OFF -- AppleNPU: OFF -- HuaweiNPU: OFF -- RKNPU: OFF -- Jetson Nano: OFF -- OpenVINO: OFF -- OpenMP: ON -- TEST: ON -- --Unit Test: OFF -- Quantization: OFF -- ModelCheck: OFF -- DEBUG: OFF -- PROFILE: OFF -- BENCHMARK: OFF -- BENCHMARK Layer: OFF -- Model Converter: OFF -- ONNX2TNN Converter: OFF -- TNN2MEM: OFF -- BENCHMARK Test Lib: OFF -- Dynamic Range Quantization: OFF -- Found PythonInterp: /usr/bin/python3 (found version "3.9.6") Generate OpenCL Source done !!!

-- Build TNN OpenCL -- Looking for C++ include unistd.h -- Looking for C++ include unistd.h - found -- Looking for C++ include stdint.h -- Looking for C++ include stdint.h - found -- Looking for C++ include inttypes.h -- Looking for C++ include inttypes.h - found -- Looking for C++ include sys/types.h -- Looking for C++ include sys/types.h - found -- Looking for C++ include sys/stat.h -- Looking for C++ include sys/stat.h - found -- Looking for C++ include fnmatch.h -- Looking for C++ include fnmatch.h - found -- Looking for C++ include stddef.h -- Looking for C++ include stddef.h - found -- Check size of uint32_t -- Check size of uint32_t - done -- Looking for strtoll -- Looking for strtoll - found -- Configuring done -- Generating done -- Build files have been written to: /Users/joseph/Projects/Mobile/TNN/scripts/build32 [ 0%] ... [ 100%] **** step 3: start build rpn arm64 **** remove build64 arm64-v8a -- android-14 is unsupported. Using minimum supported version 16. -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /Users/joseph/Downloads/android-ndk-r21b/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /Users/joseph/Downloads/android-ndk-r21b/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- The ASM compiler identification is Clang with GNU-like command-line -- Found assembler: /Users/joseph/Downloads/android-ndk-r21b/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang /Users/joseph/Projects/Mobile/TNN /Users/joseph/Projects/Mobile/TNN -- Found OpenMP_C: -fopenmp=libomp (found version "3.1") -- Found OpenMP_CXX: -fopenmp=libomp (found version "3.1") -- Found OpenMP: TRUE (found version "3.1")
-- >>>>>>>>>>>>> -- TNN BUILD INFO: -- System: Android -- Processor: aarch64 -- Cpu: ON -- X86: OFF -- Arm: ON -- Arm82: ON -- Metal: OFF -- OpenCL: ON -- CUDA: OFF -- DSP: OFF -- Atlas: OFF -- TensorRT: OFF -- AppleNPU: OFF -- HuaweiNPU: OFF -- RKNPU: OFF -- Jetson Nano: OFF -- OpenVINO: OFF -- OpenMP: ON -- TEST: ON -- --Unit Test: OFF -- Quantization: OFF -- ModelCheck: OFF -- DEBUG: OFF -- PROFILE: OFF -- BENCHMARK: OFF -- BENCHMARK Layer: OFF -- Model Converter: OFF -- ONNX2TNN Converter: OFF -- TNN2MEM: OFF -- BENCHMARK Test Lib: OFF -- Dynamic Range Quantization: OFF -- Found PythonInterp: /usr/bin/python3 (found version "3.9.6") Generate OpenCL Source done !!!

-- Build TNN OpenCL -- Looking for C++ include unistd.h -- Looking for C++ include unistd.h - found -- Looking for C++ include stdint.h -- Looking for C++ include stdint.h - found -- Looking for C++ include inttypes.h -- Looking for C++ include inttypes.h - found -- Looking for C++ include sys/types.h -- Looking for C++ include sys/types.h - found -- Looking for C++ include sys/stat.h -- Looking for C++ include sys/stat.h - found -- Looking for C++ include fnmatch.h -- Looking for C++ include fnmatch.h - found -- Looking for C++ include stddef.h -- Looking for C++ include stddef.h - found -- Check size of uint32_t -- Check size of uint32_t - done -- Looking for strtoll -- Looking for strtoll - found -- Configuring done -- Generating done -- Build files have been written to: /Users/joseph/Projects/Mobile/TNN/scripts/build64 [ 0%] ... [100%] Built target TNNTest

**** step 4: add version attr **** **** step 4: copy to release **** build done!


  **5. 详细描述bug 情况 (Describe the bug)**
I have a onnx model with a [RepVGG](https://github.com/DingXiaoH/RepVGG) backbone. I tried to convert it using both the onnx2tnn docker image and[ this web converter](https://convertmodel.com/#outputFormat=tnn).

However, I keep on getting error for wrong dimensions in a layer of the net when I tried to initialize it.
The layers are Expand (before onnxsim) and StrideSliceLayer (after onnxsim). The models work fine using onnxruntime, and the same program can use other TNN models normally.

Here are some screenshots for the structure that gave errors:
<img width="1053" alt="image" src="https://user-images.githubusercontent.com/10083616/209911455-5bbb44b2-7651-49f9-9622-1966e75eddee.png">
<img width="562" alt="image" src="https://user-images.githubusercontent.com/10083616/209911603-cf40e311-e26f-473b-a9b4-5a4042481dd8.png">
The onnxfiles and TNN files can be accessed here: https://drive.google.com/drive/folders/14WwWdDoiKdMZthk0q18Gltd0EEkT4YLn?usp=sharing

  **6. 运行日志（runtime log）**
 ```txt
 将日志粘贴在这里
[Line 36] proto content size 14069 model content size 28147104
E  virtual tnn::Status tnn::StrideSliceLayer::InferOutputShape(bool) [File source/tnn/layer/stride_slice_layer.cc][Line 44] StrideSliceLayer param got wrong size: input dims size: 2
E  virtual tnn::Status tnn::DefaultNetwork::Forward() [File source/tnn/core/default_network.cc][Line 603] Forward error code: 0x1000 msg: StrideSliceLayer param got wrong size, exit
I  virtual tnn::Status tnn::HeadPoseDetector::Init(std::shared_ptr<TNNSDKOption>) [File /Users/joseph/Projects/Mobile/FaceDetection/app/src/main/cpp/facepose/v1_0.cpp][Line 18] sample init failed code: 0x1000 msg: StrideSliceLayer param got wrong size
E  jint Java_org_japancv_keijisee_FacePose_init(JNIEnv *, jobject, jstring, jint, jint, jint) [File /Users/joseph/Projects/Mobile/FaceDetection/app/src/main/cpp/facepose/v1_0_jni.cpp][Line 59] detector init failed 4096

7. 截图（Screenshots）

 将截图粘贴在这里
 Paste screenshorts here or pastebin

1627180283 commented 1 year ago

You can try to compile the model conversion tool, and then convert the model.

iamjoseph331 commented 1 year ago

The converter was successfully compiled and also successfully converted my model, I think. The command I used: python3 onnx2tnn.py -o ~/ml/models/pose -version=v3.0 ~/ml/models/pose/v1/cloud/PoseV1-1227x1.onnx

Logs: https://pastebin.com/Fvz0YpvW However I still face an old problem

E virtual tnn::Status tnn::ArmLayerAcc::RawBuffer2ArmBlob(tnn::RawBuffer , std::shared_ptr &, tnn::BlobDesc &) [File source/tnn/device/arm/acc/arm_layer_acc.cc][Line 135] RawBuffer2ArmBlob:: unsupported buffer and blob data type: 3, 1 E virtual tnn::Status tnn::DefaultNetwork::InitLayers(tnn::NetStructure , tnn::NetResource *) [File source/tnn/core/default_network.cc][Line 321] Error Init layer /Expand (err: 4096 or 0x1000)

which claims that I am using the data type unsigned int for the buffer stated here: https://github.com/Tencent/TNN/blob/c6030afea8f85c189fd79ccc04ce684fa73f1f56/source/tnn/device/arm/acc/arm_layer_acc.cc#L135

After simplified my onnx model I got the same problem as before:

Any help would be appreciated.

gttiankai commented 1 year ago

please use TNN/tools/convert2tnn tools command:

python converter.py onnx2tnn -align=output -optimize /the/path/to/PoseV1-1227x1.onnx

TNNTest output:

D/tnn: CheckResult [File test/test.cc][Line 341] create instance success! 
D/tnn: CheckResult [File test/test.cc][Line 341] ConvertFromMat success! 
D/tnn: CheckResult [File test/test.cc][Line 341] Forward success! 
D/tnn: CheckResult [File test/test.cc][Line 341] ConvertToMat success! 
I/tnn: Print [File test/timer.cc][Line 60] PoseV1-1227x1.opt.tnnproto - ARM              TNN Benchmark time cost: min = 49.340   ms  |  max = 49.340   ms

iamjoseph331 commented 1 year ago

@gttiankai I recompiled TNN and convert2tnn tools in my x86_64 Linux machine.

Using the above command, I am able to convert the model and align it with no problem.

However in my TNNtest output I didn't receive the same results with yours.

I found out that when building the converter the ARM argument wasn't activated. In the code it looks like this is only for debug usage. I tried again with -DTNN_ARM_ENABLE="ON", but it also didn't work.

I am running TNN from the master branch and compiled it on linux_native with ARM enabled. The converted outcome is here: tnnproto tnnmodel

Tencent / TNN

ARM initializing model error: StrideSliceLayer param got wrong size: input dims size: 2 #1855