Tencent / TNN

TNN: developed by Tencent Youtu Lab and Guangying Lab, a uniform deep learning inference framework for mobile、desktop and server. TNN is distinguished by several outstanding features, including its cross-platform capability, high performance, model compression and code pruning. Based on ncnn and Rapidnet, TNN further strengthens the support and performance optimization for mobile devices, and also draws on the advantages of good extensibility and high performance from existed open source efforts. TNN has been deployed in multiple Apps from Tencent, such as Mobile QQ, Weishi, Pitu, etc. Contributions are welcome to work in collaborative with us and make TNN a better framework.
Other
4.43k stars 771 forks source link

ARM initializing model error: StrideSliceLayer param got wrong size: input dims size: 2 #1855

Open iamjoseph331 opened 1 year ago

iamjoseph331 commented 1 year ago

1. 环境(environment)

**** step 1: update version.h **** /Users/joseph/Projects/Mobile/TNN/scripts/version rm: /Users/joseph/Projects/Mobile/TNN/scripts/version/version.h: No such file or directory rm: -f: No such file or directory Target: tnn Commit Branch: master Commit Date: 2022-12-21 Commit Hash: 35690301 rm: -f: No such file or directory

**** step 2: start build rpn arm32 **** remove build32 armeabi-v7a -- android-14 is unsupported. Using minimum supported version 16. -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /Users/joseph/Downloads/android-ndk-r21b/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /Users/joseph/Downloads/android-ndk-r21b/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- The ASM compiler identification is Clang with GNU-like command-line -- Found assembler: /Users/joseph/Downloads/android-ndk-r21b/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang /Users/joseph/Projects/Mobile/TNN /Users/joseph/Projects/Mobile/TNN -- Found OpenMP_C: -fopenmp=libomp (found version "3.1") -- Found OpenMP_CXX: -fopenmp=libomp (found version "3.1") -- Found OpenMP: TRUE (found version "3.1")
-- >>>>>>>>>>>>> -- TNN BUILD INFO: -- System: Android -- Processor: armv7-a -- Cpu: ON -- X86: OFF -- Arm: ON -- Arm82: ON -- Metal: OFF -- OpenCL: ON -- CUDA: OFF -- DSP: OFF -- Atlas: OFF -- TensorRT: OFF -- AppleNPU: OFF -- HuaweiNPU: OFF -- RKNPU: OFF -- Jetson Nano: OFF -- OpenVINO: OFF -- OpenMP: ON -- TEST: ON -- --Unit Test: OFF -- Quantization: OFF -- ModelCheck: OFF -- DEBUG: OFF -- PROFILE: OFF -- BENCHMARK: OFF -- BENCHMARK Layer: OFF -- Model Converter: OFF -- ONNX2TNN Converter: OFF -- TNN2MEM: OFF -- BENCHMARK Test Lib: OFF -- Dynamic Range Quantization: OFF -- Found PythonInterp: /usr/bin/python3 (found version "3.9.6") Generate OpenCL Source done !!!

-- Build TNN OpenCL -- Looking for C++ include unistd.h -- Looking for C++ include unistd.h - found -- Looking for C++ include stdint.h -- Looking for C++ include stdint.h - found -- Looking for C++ include inttypes.h -- Looking for C++ include inttypes.h - found -- Looking for C++ include sys/types.h -- Looking for C++ include sys/types.h - found -- Looking for C++ include sys/stat.h -- Looking for C++ include sys/stat.h - found -- Looking for C++ include fnmatch.h -- Looking for C++ include fnmatch.h - found -- Looking for C++ include stddef.h -- Looking for C++ include stddef.h - found -- Check size of uint32_t -- Check size of uint32_t - done -- Looking for strtoll -- Looking for strtoll - found -- Configuring done -- Generating done -- Build files have been written to: /Users/joseph/Projects/Mobile/TNN/scripts/build32 [ 0%] ... [ 100%] **** step 3: start build rpn arm64 **** remove build64 arm64-v8a -- android-14 is unsupported. Using minimum supported version 16. -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /Users/joseph/Downloads/android-ndk-r21b/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /Users/joseph/Downloads/android-ndk-r21b/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- The ASM compiler identification is Clang with GNU-like command-line -- Found assembler: /Users/joseph/Downloads/android-ndk-r21b/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang /Users/joseph/Projects/Mobile/TNN /Users/joseph/Projects/Mobile/TNN -- Found OpenMP_C: -fopenmp=libomp (found version "3.1") -- Found OpenMP_CXX: -fopenmp=libomp (found version "3.1") -- Found OpenMP: TRUE (found version "3.1")
-- >>>>>>>>>>>>> -- TNN BUILD INFO: -- System: Android -- Processor: aarch64 -- Cpu: ON -- X86: OFF -- Arm: ON -- Arm82: ON -- Metal: OFF -- OpenCL: ON -- CUDA: OFF -- DSP: OFF -- Atlas: OFF -- TensorRT: OFF -- AppleNPU: OFF -- HuaweiNPU: OFF -- RKNPU: OFF -- Jetson Nano: OFF -- OpenVINO: OFF -- OpenMP: ON -- TEST: ON -- --Unit Test: OFF -- Quantization: OFF -- ModelCheck: OFF -- DEBUG: OFF -- PROFILE: OFF -- BENCHMARK: OFF -- BENCHMARK Layer: OFF -- Model Converter: OFF -- ONNX2TNN Converter: OFF -- TNN2MEM: OFF -- BENCHMARK Test Lib: OFF -- Dynamic Range Quantization: OFF -- Found PythonInterp: /usr/bin/python3 (found version "3.9.6") Generate OpenCL Source done !!!

-- Build TNN OpenCL -- Looking for C++ include unistd.h -- Looking for C++ include unistd.h - found -- Looking for C++ include stdint.h -- Looking for C++ include stdint.h - found -- Looking for C++ include inttypes.h -- Looking for C++ include inttypes.h - found -- Looking for C++ include sys/types.h -- Looking for C++ include sys/types.h - found -- Looking for C++ include sys/stat.h -- Looking for C++ include sys/stat.h - found -- Looking for C++ include fnmatch.h -- Looking for C++ include fnmatch.h - found -- Looking for C++ include stddef.h -- Looking for C++ include stddef.h - found -- Check size of uint32_t -- Check size of uint32_t - done -- Looking for strtoll -- Looking for strtoll - found -- Configuring done -- Generating done -- Build files have been written to: /Users/joseph/Projects/Mobile/TNN/scripts/build64 [ 0%] ... [100%] Built target TNNTest

**** step 4: add version attr **** **** step 4: copy to release **** build done!


  **5. 详细描述bug 情况 (Describe the bug)**
I have a onnx model with a [RepVGG](https://github.com/DingXiaoH/RepVGG) backbone. I tried to convert it using both the onnx2tnn docker image and[ this web converter](https://convertmodel.com/#outputFormat=tnn).

However, I keep on getting error for wrong dimensions in a layer of the net when I tried to initialize it.
The layers are Expand (before onnxsim) and StrideSliceLayer (after onnxsim). The models work fine using onnxruntime, and the same program can use other TNN models normally.

Here are some screenshots for the structure that gave errors:
<img width="1053" alt="image" src="https://user-images.githubusercontent.com/10083616/209911455-5bbb44b2-7651-49f9-9622-1966e75eddee.png">
<img width="562" alt="image" src="https://user-images.githubusercontent.com/10083616/209911603-cf40e311-e26f-473b-a9b4-5a4042481dd8.png">
The onnxfiles and TNN files can be accessed here: https://drive.google.com/drive/folders/14WwWdDoiKdMZthk0q18Gltd0EEkT4YLn?usp=sharing

  **6. 运行日志(runtime log)**
 ```txt
 将日志粘贴在这里
[Line 36] proto content size 14069 model content size 28147104
E  virtual tnn::Status tnn::StrideSliceLayer::InferOutputShape(bool) [File source/tnn/layer/stride_slice_layer.cc][Line 44] StrideSliceLayer param got wrong size: input dims size: 2
E  virtual tnn::Status tnn::DefaultNetwork::Forward() [File source/tnn/core/default_network.cc][Line 603] Forward error code: 0x1000 msg: StrideSliceLayer param got wrong size, exit
I  virtual tnn::Status tnn::HeadPoseDetector::Init(std::shared_ptr<TNNSDKOption>) [File /Users/joseph/Projects/Mobile/FaceDetection/app/src/main/cpp/facepose/v1_0.cpp][Line 18] sample init failed code: 0x1000 msg: StrideSliceLayer param got wrong size
E  jint Java_org_japancv_keijisee_FacePose_init(JNIEnv *, jobject, jstring, jint, jint, jint) [File /Users/joseph/Projects/Mobile/FaceDetection/app/src/main/cpp/facepose/v1_0_jni.cpp][Line 59] detector init failed 4096

7. 截图(Screenshots)

 将截图粘贴在这里
 Paste screenshorts here or pastebin
image
1627180283 commented 1 year ago

You can try to compile the model conversion tool, and then convert the model.

iamjoseph331 commented 1 year ago

The converter was successfully compiled and also successfully converted my model, I think. The command I used: python3 onnx2tnn.py -o ~/ml/models/pose -version=v3.0 ~/ml/models/pose/v1/cloud/PoseV1-1227x1.onnx

Logs: https://pastebin.com/Fvz0YpvW However I still face an old problem

E virtual tnn::Status tnn::ArmLayerAcc::RawBuffer2ArmBlob(tnn::RawBuffer , std::shared_ptr &, tnn::BlobDesc &) [File source/tnn/device/arm/acc/arm_layer_acc.cc][Line 135] RawBuffer2ArmBlob:: unsupported buffer and blob data type: 3, 1 E virtual tnn::Status tnn::DefaultNetwork::InitLayers(tnn::NetStructure , tnn::NetResource *) [File source/tnn/core/default_network.cc][Line 321] Error Init layer /Expand (err: 4096 or 0x1000)

image

which claims that I am using the data type unsigned int for the buffer stated here: https://github.com/Tencent/TNN/blob/c6030afea8f85c189fd79ccc04ce684fa73f1f56/source/tnn/device/arm/acc/arm_layer_acc.cc#L135

After simplified my onnx model I got the same problem as before: image

Any help would be appreciated.

gttiankai commented 1 year ago
  1. please use TNN/tools/convert2tnn tools command:
    python converter.py onnx2tnn -align=output -optimize /the/path/to/PoseV1-1227x1.onnx
  2. TNNTest output:
    D/tnn: CheckResult [File test/test.cc][Line 341] create instance success! 
    D/tnn: CheckResult [File test/test.cc][Line 341] ConvertFromMat success! 
    D/tnn: CheckResult [File test/test.cc][Line 341] Forward success! 
    D/tnn: CheckResult [File test/test.cc][Line 341] ConvertToMat success! 
    I/tnn: Print [File test/timer.cc][Line 60] PoseV1-1227x1.opt.tnnproto - ARM              TNN Benchmark time cost: min = 49.340   ms  |  max = 49.340   ms 
iamjoseph331 commented 1 year ago

@gttiankai I recompiled TNN and convert2tnn tools in my x86_64 Linux machine.

Using the above command, I am able to convert the model and align it with no problem.

image

However in my TNNtest output I didn't receive the same results with yours.

image

I found out that when building the converter the ARM argument wasn't activated. In the code it looks like this is only for debug usage. I tried again with -DTNN_ARM_ENABLE="ON", but it also didn't work.

image

I am running TNN from the master branch and compiled it on linux_native with ARM enabled. The converted outcome is here: tnnproto tnnmodel