Different result even if input is a zeros mat

[Amadeus-AI]

I got different result between onnx and ncnn model even if input is a zeros mat. (Already checked the FAQ) model.zip Password:0000 Here's the onnx and ncnn model. (You can also try to regenerate ncnn model from onnx) @nihui

A simple test:

#include "net.h"

int main(int argc, char** argv)
{
    ncnn::UnlockedPoolAllocator ncnn_blob_pool_allocator_;
    ncnn::PoolAllocator ncnn_workspace_pool_allocator_;
    ncnn::Net ncnn_detector_;

    ncnn::Option opt;
    opt.blob_allocator = &ncnn_blob_pool_allocator_;
    opt.workspace_allocator = &ncnn_workspace_pool_allocator_;

    ncnn_detector_.opt = opt;
    ncnn_detector_.load_param("model-opt.param");
    ncnn_detector_.load_model("model-opt.bin");

    cv::Mat ncnn_cv_mat(56, 56, CV_32FC3, cv::Scalar(0, 0, 0));
    ncnn::Mat ncnn_in = ncnn::Mat::from_pixels(ncnn_cv_mat.data, ncnn::Mat::PIXEL_BGR, ncnn_cv_mat.cols, ncnn_cv_mat.rows);

    auto ncnn_extractor = ncnn_detector_.create_extractor();
    ncnn_extractor.input("main_input", in);

    ncnn::Mat out;
    ncnn_extractor.extract("class_ret", out);

    std::vector<float> cls_scores;
    cls_scores.resize(ncnn_out.w);
    for (int j = 0; j < ncnn_out.w; j++)
    {
        cls_scores[j] = ncnn_out[j];
    }

    return 0;
}

NCNN output: 0.43, -0.66

import onnxruntime as ort
import numpy as np
from PIL import Image

img = Image.new("RGB", (56, 56), "black")
inputs = np.array(img, dtype=np.float32)
inputs = inputs.transpose(2, 0, 1)
inputs = np.expand_dims(inputs, axis=0)

ort_sess = ort.InferenceSession('model.onnx')
outputs = ort_sess.run(None, {'main_input': inputs })
print(outputs)

ONNX(and other transform like tflite): -0.05 , -0.09

on win10 x64, tested every version from 2020.12 till latest version of ncnn

[nihui]

I got password error when decompressing your model.zip

[nihui]

Your onnx file seems to be converted from tensorflow graph, while onnx2ncnn supports onnx exported from pytorch. Try keras2ncnn or mlir2ncnn for tensorflow model conversion

[Amadeus-AI]

@nihui My onnx is exported from pytorch

[Amadeus-AI]

I found that maybe the following operation is the cause:

class Conv2d_cd(nn.Module): 
    def __init__(self, in_channels, out_channels, kernel_size=3, stride=1, padding=1, dilation=1, groups=1, bias=False, theta=0.5):

        super(Conv2d_cd, self).__init__() 
        self.conv = nn.Conv2d(in_channels, out_channels, kernel_size=kernel_size, stride=stride, padding=padding, dilation=dilation, groups=groups, bias=bias)
        self.theta = theta

    def forward(self, x):
        out_normal = self.conv(x)

        kernel_diff = self.conv.weight.sum(2).sum(2)
        out_diff = torch.matmul(x.permute(0,2,3,1), kernel_diff.permute(1,0)).permute(0,3,1,2)

        return out_normal - self.theta * out_diff

You can test this module

[Amadeus-AI]

Is there any update on this issue? @nihui

[nihui]

针对onnx模型转换的各种问题，推荐使用最新的pnnx工具转换到ncnn In view of various problems in onnx model conversion, it is recommended to use the latest pnnx tool to convert your model to ncnn

pip install pnnx
pnnx model.onnx inputshape=[1,3,224,224]

详细参考文档 Detailed reference documentation https://github.com/pnnx/pnnx https://github.com/Tencent/ncnn/wiki/use-ncnn-with-pytorch-or-onnx#how-to-use-pnnx