pytorch转ONNX模型转MNN模型，同一张图片推理结果不一致

[alexander2618]

Github版本:

Github Version:2.1/2.4

编译方式:

Compiling Method

git clone https://github.com/alibaba/MNN/
cd /path/to/MNN
./schema/generate.sh
mkdir build && cd build
cmake .. -DMNN_BUILD_CONVERTER=ON -DMNN_CUDA=ON -DMNN_OPENCL=ON -DMNN_BUILD_SHARED_LIBS=OFF -DMNN_SEP_BUILD=OFF

模型转换日志日志:

converter Log:

The device support dot:0, support fp16:0, support i8mm: 0
Start to Convert Other Model Format To MNN Model...
[00:54:57] /home/MNN_20211029_PadChannel_Static/authen_mnn_2.4/tools/converter/source/onnx/onnxConverter.cpp:40: ONNX Model ir version: 7
Start to Optimize the MNN Net...
232 op name is empty or dup, set to Const232
249 op name is empty or dup, set to BinaryOp249
347 op name is empty or dup, set to Unsqueeze347
577 op name is empty or dup, set to BinaryOp577
589 op name is empty or dup, set to Unsqueeze589
601 op name is empty or dup, set to BinaryOp601
621 op name is empty or dup, set to BinaryOp621
630 op name is empty or dup, set to Unsqueeze630
816 op name is empty or dup, set to Const816
972 op name is empty or dup, set to BinaryOp972
1022 op name is empty or dup, set to BinaryOp1022
1091 op name is empty or dup, set to BinaryOp1091
1104 op name is empty or dup, set to BinaryOp1104
1109 op name is empty or dup, set to Const1109
1123 op name is empty or dup, set to Const1123
1183 op name is empty or dup, set to BinaryOp1183
1199 op name is empty or dup, set to BinaryOp1199
1316 op name is empty or dup, set to Shape1316
1324 op name is empty or dup, set to BinaryOp1324
1333 op name is empty or dup, set to Unsqueeze1333
1535 op name is empty or dup, set to BinaryOp1535
1536 op name is empty or dup, set to Unsqueeze1536
1685 op name is empty or dup, set to Unsqueeze1685
1687 op name is empty or dup, set to BinaryOp1687
1937 op name is empty or dup, set to BinaryOp1937
2082 op name is empty or dup, set to BinaryOp2082
2085 op name is empty or dup, set to BinaryOp2085
2086 op name is empty or dup, set to Unsqueeze2086
2087 op name is empty or dup, set to Const2087
2088 op name is empty or dup, set to StridedSlice2088
2090 op name is empty or dup, set to BinaryOp2090
2113 op name is empty or dup, set to BinaryOp2113
2114 op name is empty or dup, set to Unsqueeze2114
2115 op name is empty or dup, set to Const2115
2261 op name is empty or dup, set to BinaryOp2261
2309 op name is empty or dup, set to BinaryOp2309
2342 op name is empty or dup, set to Unsqueeze2342
2374 op name is empty or dup, set to Unsqueeze2374
2451 op name is empty or dup, set to BinaryOp2451
2714 op name is empty or dup, set to Unsqueeze2714
2725 op name is empty or dup, set to BinaryOp2725
2726 op name is empty or dup, set to BinaryOp2726
2743 op name is empty or dup, set to BinaryOp2743
2769 op name is empty or dup, set to BinaryOp2769
2855 op name is empty or dup, set to Unsqueeze2855
2862 op name is empty or dup, set to BinaryOp2862
2916 op name is empty or dup, set to BinaryOp2916
2920 op name is empty or dup, set to Unsqueeze2920
2938 op name is empty or dup, set to Unsqueeze2938
2942 op name is empty or dup, set to BinaryOp2942
3083 op name is empty or dup, set to BinaryOp3083
3084 op name is empty or dup, set to Unsqueeze3084
3085 op name is empty or dup, set to Const3085
3086 op name is empty or dup, set to StridedSlice3086
3088 op name is empty or dup, set to BinaryOp3088
3291 op name is empty or dup, set to BinaryOp3291
3455 op name is empty or dup, set to BinaryOp3455
3929 op name is empty or dup, set to Unsqueeze3929
3940 op name is empty or dup, set to BinaryOp3940
3946 op name is empty or dup, set to Unsqueeze3946
3964 op name is empty or dup, set to Unsqueeze3964
3966 op name is empty or dup, set to BinaryOp3966
4002 op name is empty or dup, set to Unsqueeze4002
4021 op name is empty or dup, set to Unsqueeze4021
4108 op name is empty or dup, set to Unsqueeze4108
4168 op name is empty or dup, set to Unsqueeze4168
4288 op name is empty or dup, set to Unsqueeze4288
4290 op name is empty or dup, set to BinaryOp4290
4363 op name is empty or dup, set to Unsqueeze4363
4365 op name is empty or dup, set to Unsqueeze4365
4366 op name is empty or dup, set to Const4366
4373 op name is empty or dup, set to BinaryOp4373
4378 op name is empty or dup, set to Const4378
4467 op name is empty or dup, set to BinaryOp4467
4480 op name is empty or dup, set to Shape4480
4487 op name is empty or dup, set to BinaryOp4487
4488 op name is empty or dup, set to BinaryOp4488
4505 op name is empty or dup, set to BinaryOp4505
4775 op name is empty or dup, set to BinaryOp4775
4876 op name is empty or dup, set to Unsqueeze4876
4887 op name is empty or dup, set to BinaryOp4887
4888 op name is empty or dup, set to Unsqueeze4888
4982 op name is empty or dup, set to BinaryOp4982
5134 op name is empty or dup, set to Unsqueeze5134
5145 op name is empty or dup, set to BinaryOp5145
5146 op name is empty or dup, set to Unsqueeze5146
5169 op name is empty or dup, set to BinaryOp5169
5323 op name is empty or dup, set to BinaryOp5323
5340 op name is empty or dup, set to BinaryOp5340
5675 op name is empty or dup, set to Unsqueeze5675
5677 op name is empty or dup, set to Unsqueeze5677
5678 op name is empty or dup, set to Const5678
5805 op name is empty or dup, set to BinaryOp5805
5809 op name is empty or dup, set to Unsqueeze5809
5905 op name is empty or dup, set to Const5905
5914 op name is empty or dup, set to Unsqueeze5914
5921 op name is empty or dup, set to BinaryOp5921
5933 op name is empty or dup, set to Unsqueeze5933
6132 op name is empty or dup, set to BinaryOp6132
6158 op name is empty or dup, set to BinaryOp6158
6229 op name is empty or dup, set to BinaryOp6229
6251 op name is empty or dup, set to BinaryOp6251
inputTensors : [ input, ]
outputTensors: [ output, ]
Converted Success!

使用testMNNFromONNX.py测试mnn模型

使用2.1版本和2.4版本测试结果不一致

version 2.1

Dir exist
onnx/test.onnx
tensor(float)
['output']
inputs:
input
onnx/
outputs:
onnx/output.txt (1, 1536)
onnx/
Start to Convert Other Model Format To MNN Model...
[01:26:55] /home/MNN_20211029_PadChannel_Static/authen_mnn/tools/converter/source/onnx/onnxConverter.cpp:40: ONNX Model ir version: 7
Start to Optimize the MNN Net...
131 op name is empty or dup, set to Unsqueeze131
133 op name is empty or dup, set to Unsqueeze133
135 op name is empty or dup, set to Unsqueeze135
381 op name is empty or dup, set to Const381
468 op name is empty or dup, set to Unsqueeze468
477 op name is empty or dup, set to BinaryOp477
600 op name is empty or dup, set to Unsqueeze600
612 op name is empty or dup, set to BinaryOp612
621 op name is empty or dup, set to Unsqueeze621
865 op name is empty or dup, set to Const865
1032 op name is empty or dup, set to BinaryOp1032
1261 op name is empty or dup, set to Unsqueeze1261
1437 op name is empty or dup, set to BinaryOp1437
1795 op name is empty or dup, set to BinaryOp1795
1884 op name is empty or dup, set to Const1884
2071 op name is empty or dup, set to Const2071
2326 op name is empty or dup, set to BinaryOp2326
2352 op name is empty or dup, set to Unsqueeze2352
2354 op name is empty or dup, set to BinaryOp2354
2431 op name is empty or dup, set to Unsqueeze2431
2432 op name is empty or dup, set to Const2432
2442 op name is empty or dup, set to Unsqueeze2442
2512 op name is empty or dup, set to Unsqueeze2512
2573 op name is empty or dup, set to BinaryOp2573
2736 op name is empty or dup, set to Const2736
2907 op name is empty or dup, set to BinaryOp2907
2908 op name is empty or dup, set to BinaryOp2908
2991 op name is empty or dup, set to Const2991
2994 op name is empty or dup, set to BinaryOp2994
3085 op name is empty or dup, set to BinaryOp3085
3113 op name is empty or dup, set to BinaryOp3113
3426 op name is empty or dup, set to Unsqueeze3426
3435 op name is empty or dup, set to BinaryOp3435
3532 op name is empty or dup, set to BinaryOp3532
3536 op name is empty or dup, set to Unsqueeze3536
3537 op name is empty or dup, set to Const3537
3556 op name is empty or dup, set to Unsqueeze3556
3585 op name is empty or dup, set to Unsqueeze3585
3586 op name is empty or dup, set to Const3586
3617 op name is empty or dup, set to BinaryOp3617
3638 op name is empty or dup, set to Rank3638
3799 op name is empty or dup, set to Const3799
3802 op name is empty or dup, set to BinaryOp3802
3832 op name is empty or dup, set to BinaryOp3832
3855 op name is empty or dup, set to BinaryOp3855
4082 op name is empty or dup, set to Unsqueeze4082
4139 op name is empty or dup, set to Unsqueeze4139
4185 op name is empty or dup, set to Unsqueeze4185
4202 op name is empty or dup, set to Unsqueeze4202
4243 op name is empty or dup, set to Unsqueeze4243
4731 op name is empty or dup, set to Unsqueeze4731
4888 op name is empty or dup, set to Unsqueeze4888
4890 op name is empty or dup, set to Unsqueeze4890
4903 op name is empty or dup, set to Unsqueeze4903
4915 op name is empty or dup, set to BinaryOp4915
4959 op name is empty or dup, set to Unsqueeze4959
4972 op name is empty or dup, set to Unsqueeze4972
5072 op name is empty or dup, set to BinaryOp5072
5082 op name is empty or dup, set to Unsqueeze5082
5099 op name is empty or dup, set to Unsqueeze5099
5101 op name is empty or dup, set to BinaryOp5101
5144 op name is empty or dup, set to BinaryOp5144
5161 op name is empty or dup, set to Unsqueeze5161
5172 op name is empty or dup, set to Unsqueeze5172
5234 op name is empty or dup, set to Unsqueeze5234
5347 op name is empty or dup, set to Const5347
5596 op name is empty or dup, set to Shape5596
5604 op name is empty or dup, set to BinaryOp5604
5628 op name is empty or dup, set to Unsqueeze5628
5632 op name is empty or dup, set to BinaryOp5632
5684 op name is empty or dup, set to BinaryOp5684
5707 op name is empty or dup, set to Unsqueeze5707
5708 op name is empty or dup, set to Const5708
5717 op name is empty or dup, set to BinaryOp5717
5718 op name is empty or dup, set to Unsqueeze5718
5740 op name is empty or dup, set to Unsqueeze5740
5893 op name is empty or dup, set to Const5893
6174 op name is empty or dup, set to Unsqueeze6174
6186 op name is empty or dup, set to BinaryOp6186
6264 op name is empty or dup, set to Unsqueeze6264
6363 op name is empty or dup, set to BinaryOp6363
6394 op name is empty or dup, set to BinaryOp6394
6483 op name is empty or dup, set to Unsqueeze6483
6553 op name is empty or dup, set to BinaryOp6553
6572 op name is empty or dup, set to BinaryOp6572
6604 op name is empty or dup, set to Unsqueeze6604
6634 op name is empty or dup, set to BinaryOp6634
6696 op name is empty or dup, set to Unsqueeze6696
6731 op name is empty or dup, set to Unsqueeze6731
6746 op name is empty or dup, set to Unsqueeze6746
6762 op name is empty or dup, set to Unsqueeze6762
6857 op name is empty or dup, set to Unsqueeze6857
6967 op name is empty or dup, set to Const6967
inputTensors : [ input, ]
outputTensors: [ output, ]
Converted Success!
Check convert result by onnx, thredhold is 0.01
input
output: output
output: (1, 1536, )
TEST_SUCCESS

version 2.4

onnx/test.onnx
tensor(float)
['output']
inputs:
input
onnx/
outputs:
onnx/output.txt (1, 1536)
onnx/
The device support dot:0, support fp16:0, support i8mm: 0
Start to Convert Other Model Format To MNN Model...
[01:24:30] /home/MNN_20211029_PadChannel_Static/authen_mnn_2.4/tools/converter/source/onnx/onnxConverter.cpp:40: ONNX Model ir version: 7
Start to Optimize the MNN Net...
233 op name is empty or dup, set to Const233
250 op name is empty or dup, set to BinaryOp250
348 op name is empty or dup, set to Unsqueeze348
578 op name is empty or dup, set to BinaryOp578
590 op name is empty or dup, set to Unsqueeze590
602 op name is empty or dup, set to BinaryOp602
623 op name is empty or dup, set to BinaryOp623
640 op name is empty or dup, set to BinaryOp640
817 op name is empty or dup, set to Const817
973 op name is empty or dup, set to BinaryOp973
1024 op name is empty or dup, set to BinaryOp1024
1105 op name is empty or dup, set to BinaryOp1105
1110 op name is empty or dup, set to Const1110
1200 op name is empty or dup, set to BinaryOp1200
1317 op name is empty or dup, set to Shape1317
1325 op name is empty or dup, set to BinaryOp1325
1334 op name is empty or dup, set to Unsqueeze1334
1536 op name is empty or dup, set to BinaryOp1536
1537 op name is empty or dup, set to Unsqueeze1537
1686 op name is empty or dup, set to Unsqueeze1686
1688 op name is empty or dup, set to BinaryOp1688
1799 op name is empty or dup, set to Const1799
1938 op name is empty or dup, set to BinaryOp1938
2084 op name is empty or dup, set to BinaryOp2084
2086 op name is empty or dup, set to BinaryOp2086
2087 op name is empty or dup, set to Unsqueeze2087
2088 op name is empty or dup, set to Const2088
2089 op name is empty or dup, set to StridedSlice2089
2091 op name is empty or dup, set to BinaryOp2091
2118 op name is empty or dup, set to StridedSlice2118
2120 op name is empty or dup, set to BinaryOp2120
2262 op name is empty or dup, set to BinaryOp2262
2343 op name is empty or dup, set to Unsqueeze2343
2452 op name is empty or dup, set to BinaryOp2452
2486 op name is empty or dup, set to BinaryOp2486
2715 op name is empty or dup, set to Unsqueeze2715
2727 op name is empty or dup, set to BinaryOp2727
2744 op name is empty or dup, set to BinaryOp2744
2770 op name is empty or dup, set to BinaryOp2770
2856 op name is empty or dup, set to Unsqueeze2856
2863 op name is empty or dup, set to BinaryOp2863
2917 op name is empty or dup, set to BinaryOp2917
2921 op name is empty or dup, set to Unsqueeze2921
2939 op name is empty or dup, set to Unsqueeze2939
2943 op name is empty or dup, set to BinaryOp2943
2980 op name is empty or dup, set to Unsqueeze2980
3084 op name is empty or dup, set to BinaryOp3084
3085 op name is empty or dup, set to Unsqueeze3085
3086 op name is empty or dup, set to Const3086
3087 op name is empty or dup, set to StridedSlice3087
3089 op name is empty or dup, set to BinaryOp3089
3129 op name is empty or dup, set to Unsqueeze3129
3175 op name is empty or dup, set to Unsqueeze3175
3282 op name is empty or dup, set to BinaryOp3282
3292 op name is empty or dup, set to BinaryOp3292
3340 op name is empty or dup, set to Unsqueeze3340
3456 op name is empty or dup, set to BinaryOp3456
3506 op name is empty or dup, set to BinaryOp3506
3930 op name is empty or dup, set to Unsqueeze3930
3941 op name is empty or dup, set to BinaryOp3941
3947 op name is empty or dup, set to Unsqueeze3947
3965 op name is empty or dup, set to Unsqueeze3965
3967 op name is empty or dup, set to BinaryOp3967
4025 op name is empty or dup, set to BinaryOp4025
4109 op name is empty or dup, set to Unsqueeze4109
4289 op name is empty or dup, set to Unsqueeze4289
4291 op name is empty or dup, set to BinaryOp4291
4353 op name is empty or dup, set to Unsqueeze4353
4364 op name is empty or dup, set to Unsqueeze4364
4366 op name is empty or dup, set to Unsqueeze4366
4367 op name is empty or dup, set to Const4367
4374 op name is empty or dup, set to BinaryOp4374
4379 op name is empty or dup, set to Const4379
4431 op name is empty or dup, set to Unsqueeze4431
4432 op name is empty or dup, set to Const4432
4469 op name is empty or dup, set to BinaryOp4469
4481 op name is empty or dup, set to Shape4481
4489 op name is empty or dup, set to BinaryOp4489
4506 op name is empty or dup, set to BinaryOp4506
4776 op name is empty or dup, set to BinaryOp4776
4877 op name is empty or dup, set to Unsqueeze4877
4888 op name is empty or dup, set to BinaryOp4888
4889 op name is empty or dup, set to Unsqueeze4889
4984 op name is empty or dup, set to BinaryOp4984
5135 op name is empty or dup, set to Unsqueeze5135
5136 op name is empty or dup, set to BinaryOp5136
5146 op name is empty or dup, set to BinaryOp5146
5147 op name is empty or dup, set to Unsqueeze5147
5148 op name is empty or dup, set to BinaryOp5148
5170 op name is empty or dup, set to BinaryOp5170
5324 op name is empty or dup, set to BinaryOp5324
5341 op name is empty or dup, set to BinaryOp5341
5676 op name is empty or dup, set to Unsqueeze5676
5678 op name is empty or dup, set to Unsqueeze5678  
5679 op name is empty or dup, set to Const5679
5762 op name is empty or dup, set to Unsqueeze5762
5764 op name is empty or dup, set to Unsqueeze5764
5806 op name is empty or dup, set to BinaryOp5806
5810 op name is empty or dup, set to Unsqueeze5810
5905 op name is empty or dup, set to Unsqueeze5905
5906 op name is empty or dup, set to Const5906
5914 op name is empty or dup, set to BinaryOp5914
5915 op name is empty or dup, set to Unsqueeze5915
5922 op name is empty or dup, set to BinaryOp5922
5934 op name is empty or dup, set to Unsqueeze5934
6159 op name is empty or dup, set to BinaryOp6159
6230 op name is empty or dup, set to BinaryOp6230  
6252 op name is empty or dup, set to BinaryOp6252  
inputTensors : [ input, ]  
outputTensors: [ output, ]  
Converted Success!  
Check convert result by onnx, thredhold is 0.01  
input  
output: output  
output: (1, 1536, )  
TESTERROR output value error : absMaxV:0.039442 - DiffMax 0.000455  
Error for output output  
Save mnn result to  .error director  

推理代码如下

#include "ExtractorEx.h"
#include "cropImage.h"
using namespace cv;

ExtractorEx::ExtractorEx()
{
    std::string extractModelDirPath = "../../models/extractEx_swim.pkg";
    MNN::ScheduleConfig config;
    config.numThread = 1;
    config.type = MNN_FORWARD_CUDA;
    MNN::BackendConfig backendConfig;
    backendConfig.precision = MNN::BackendConfig::PrecisionMode::Precision_Low;
    config.backendConfig = &backendConfig;

    extract_interpreter = std::shared_ptr<MNN::Interpreter>(MNN::Interpreter::createFromFile(extractModelDirPath.c_str()));
    extract_session = extract_interpreter->createSession(config);
    extract_input_tensor = extract_interpreter->getSessionInput(extract_session, nullptr);

    std::vector<int> dimVector{ 1, 3, 224, 224 };
    extract_interpreter->resizeTensor(extract_input_tensor, dimVector);
    extract_interpreter->resizeSession(extract_session);

    std::shared_ptr<MNN::CV::ImageProcess> pretreat(
        MNN::CV::ImageProcess::create(MNN::CV::BGR, MNN::CV::RGB, extract_mean_vals, 3,
            extract_norm_vals, 3));

    dfeature = new float[1536];
}

ExtractorEx::~ExtractorEx()
{
    delete[] dfeature;
    if (extract_interpreter != nullptr) {
        extract_interpreter->releaseModel();
        extract_interpreter->releaseSession(extract_session);
    }
}

float* ExtractorEx::extractFeaturesDirect(Mat miniBGR1, Mat miniBGR2)
{
    std::shared_ptr<MNN::CV::ImageProcess> pretreat(
        MNN::CV::ImageProcess::create(MNN::CV::BGR, MNN::CV::RGB, extract_mean_vals, 3,
            extract_norm_vals, 3));

    cout << "direct" << endl;
    float* feature;
    pretreat->convert(miniBGR1.data, 224, 224, miniBGR1.step[0], extract_input_tensor);
    feature = inference();
    pretreat->convert(miniBGR2.data, 224, 224, miniBGR2.step[0], extract_input_tensor);
    feature = inference();
    pretreat->convert(miniBGR1.data, 224, 224, miniBGR1.step[0], extract_input_tensor);
    feature = inference();
    return feature;
}

float* ExtractorEx::inference(){
    // run network
    printf("input tensor\n");
    extract_input_tensor->print();
    MNN::ErrorCode error =  extract_interpreter->runSession(extract_session);
    cout << "errorcode: " << error << endl;

    MNN::Tensor* tensor_detection_outs = extract_interpreter->getSessionOutput(extract_session, NULL);
    printf("output tensor\n");
    tensor_detection_outs->print();
    MNN::Tensor output(tensor_detection_outs, MNN::Tensor::CAFFE);
    tensor_detection_outs->copyToHostTensor(&output);

    memset((void*)dfeature, 0, 1536 * sizeof(float));
    printf("before\n");
    for (int i = 0; i != 1536; i++) {
        if(i < 10)
            printf("%d, %f\n", i, dfeature[i]);
    }

    printf("after\n");
    double normTemp = 0.0f;
    for (int i = 0; i != 1536; i++) {
        dfeature[i] = output.host<float>()[i];
        if(i < 10)
            printf("%d, %f\n", i, dfeature[i]);
        // normTemp += dfeature[i] * dfeature[i];
    }
    return dfeature;
}

int main(){
Mat image = imread("1.png");

ExtractorEx* extractor = new ExtractorEx();
cout << "ExtractorEx Engin e Successed" << endl;

float* fea;
fea = extractor->extractFeaturesDirect(image, image.clone());

delete extractor;
}

运行日志

output2.log

现象

在inference输入了两张一样的BGR数据（第二张由第一张clone），进行了三次推理，发现即时同一张图像数据，推理的结果不一致，且多次运行结果表现得较为随机，详情可以见log日志。 打印了输入Tensor，三次数据确定数据一致。

[wangzhaode]

ONNX模型是否可以提供一下？

[alexander2618]

ONNX模型是否可以提供一下？

回复错地方了，我钉钉上发给您

[wangzhaode]

方便的话直接在这里上传也可以

[alexander2618]

方便的话直接在这里上传也可以

文件太大了，只能麻烦您下载一下了 https://qr.dingtalk.com/page/yunpan?route=previewDentry&spaceId=21211392472&fileId=98837360217&type=file

[wangzhaode]

方便的话直接在这里上传也可以

文件太大了，只能麻烦您下载一下了 https://qr.dingtalk.com/page/yunpan?route=previewDentry&spaceId=21211392472&fileId=98837360217&type=file

您这个链接不对吧

[alexander2618]

方便的话直接在这里上传也可以

文件太大了，只能麻烦您下载一下了 https://qr.dingtalk.com/page/yunpan?route=previewDentry&spaceId=21211392472&fileId=98837360217&type=file

您这个链接不对吧

下载 这样可以吗

[wangzhaode]

可以下载了 我看一下

[github-actions[bot]]

Marking as stale. No activity in 60 days.