Resnest50 to onnx. Different inference results

Description

I converted resnest50 model according to 19808. Now I try to compare results given by onnx model with onnxruntime. The difference between them is quite large. The output of numpy.testing.assert_allclose is

Mismatched elements: 1000 / 1000 (100%)
Max absolute difference: 0.32469702
Max relative difference: 27.369648

Error Message

To Reproduce

Steps to reproduce

model conversion

from gluoncv import model_zoo
import numpy as np
import mxnet as mx
model_name = 'resnest50'
resnet50 = model_zoo.get_model(model_name, pretrained=True)
print(model_name+' downloaded')
resnet50.hybridize()
print(model_name+' hybridized')
input_shape=(1,3,224,224)
data_array = np.random.uniform(0, 1, size=input_shape).astype("float32")
mx_data = mx.nd.array(data_array)
resnet50(mx_data)
resnet50.export(model_name)
print(model_name+' exported')
from mxnet.contrib import onnx as onnx_mxnet
onnx_file='./tp.onnx'
params = './'+model_name+'-0000.params'
sym='./'+model_name+'-symbol.json'
onnx_mxnet.export_model(sym, params, [input_shape], np.float32, onnx_file)
print('onnx export done')

Model testing


import onnxruntime as rt
import numpy
from onnxruntime.datasets import get_example

sess = rt.InferenceSession('tp.onnx')
input_name = sess.get_inputs()[0].name
data_array = np.random.uniform(0, 1, size=input_shape).astype("float32")
mx_data = mx.nd.array(data_array)
onnx_data = mx_data.asnumpy()
a = sess.run(None, {input_name: onnx_data})[0][0]
b = resnet50(mx_data)[0].asnumpy()
print(numpy.testing.assert_allclose(b,a))

Environment

Environment Information

----------Python Info---------- Version : 3.7.10 Compiler : GCC 9.3.0 Build : ('default', 'Feb 20 2021 21:15:28') Arch : ('64bit', '') ------------Pip Info----------- Version : 20.0.2 Directory : /usr/lib/python3/dist-packages/pip ----------MXNet Info----------- Version : 1.7.0 Directory : /home/local/.local/lib/python3.7/site-packages/mxnet Commit Hash : 64f737cdd59fe88d2c5b479f25d011c5156b6a8a 64f737cdd59fe88d2c5b479f25d011c5156b6a8a 64f737cdd59fe88d2c5b479f25d011c5156b6a8a 64f737cdd59fe88d2c5b479f25d011c5156b6a8a 64f737cdd59fe88d2c5b479f25d011c5156b6a8a 64f737cdd59fe88d2c5b479f25d011c5156b6a8a 64f737cdd59fe88d2c5b479f25d011c5156b6a8a 64f737cdd59fe88d2c5b479f25d011c5156b6a8a 64f737cdd59fe88d2c5b479f25d011c5156b6a8a 64f737cdd59fe88d2c5b479f25d011c5156b6a8a Library : ['/home/local/.local/lib/python3.7/site-packages/mxnet/libmxnet.so'] Build features: ✔ CUDA ✔ CUDNN ✔ NCCL ✔ CUDA_RTC ✖ TENSORRT ✔ CPU_SSE ✔ CPU_SSE2 ✔ CPU_SSE3 ✔ CPU_SSE4_1 ✔ CPU_SSE4_2 ✖ CPU_SSE4A ✔ CPU_AVX ✖ CPU_AVX2 ✔ OPENMP ✖ SSE ✔ F16C ✖ JEMALLOC ✔ BLAS_OPEN ✖ BLAS_ATLAS ✖ BLAS_MKL ✖ BLAS_APPLE ✔ LAPACK ✔ MKLDNN ✔ OPENCV ✖ CAFFE ✖ PROFILER ✔ DIST_KVSTORE ✖ CXX14 ✖ INT64_TENSOR_SIZE ✔ SIGNAL_HANDLER ✖ DEBUG ✖ TVM_OP ----------System Info---------- Platform : Linux-5.8.0-53-generic-x86_64-with-Ubuntu-20.04-focal system : Linux node : tva-pc-03 release : 5.8.0-53-generic version : #60~20.04.1-Ubuntu SMP Thu May 6 09:52:46 UTC 2021 ----------Hardware Info---------- machine : x86_64 processor : x86_64 Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian Address sizes: 39 bits physical, 48 bits virtual CPU(s): 12 On-line CPU(s) list: 0-11 Thread(s) per core: 2 Core(s) per socket: 6 Socket(s): 1 NUMA node(s): 1 Vendor ID: GenuineIntel CPU family: 6 Model: 165 Model name: Intel(R) Core(TM) i5-10600K CPU @ 4.10GHz Stepping: 5 CPU MHz: 4399.823 CPU max MHz: 4800,0000 CPU min MHz: 800,0000 BogoMIPS: 8199.79 Virtualization: VT-x L1d cache: 192 KiB L1i cache: 192 KiB L2 cache: 1,5 MiB L3 cache: 12 MiB NUMA node0 CPU(s): 0-11 Vulnerability Itlb multihit: KVM: Mitigation: VMX disabled Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Spec store bypass: Vulnerable Vulnerability Spectre v1: Vulnerable: __user pointer sanitization and usercopy barriers only; no swapgs barriers Vulnerability Spectre v2: Vulnerable, IBPB: disabled, STIBP: disabled Vulnerability Srbds: Not affected Vulnerability Tsx async abort: Not affected Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse s se2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtop ology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma c x16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_ lm abm 3dnowprefetch cpuid_fault epb invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap c lflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp pku ospke md_clear flush_l1d arch_capabilities ----------Network Test---------- Setting timeout: 10 Timing for MXNet: https://github.com/apache/incubator-mxnet, DNS: 0.0006 sec, LOAD: 0.4579 sec. Timing for Gluon Tutorial(en): http://gluon.mxnet.io, DNS: 0.0247 sec, LOAD: 0.1638 sec. Error open Gluon Tutorial(cn): https://zh.gluon.ai, , DNS finished in 0.036272525787353516 sec. Timing for FashionMNIST: https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/dataset/fashion-mnist/train-labels-idx1-ubyte.gz, DNS: 0.1557 sec, LOAD: 1.3185 sec. Timing for PYPI: https://pypi.python.org/pypi/pip, DNS: 0.1053 sec, LOAD: 0.6913 sec. Error open Conda: https://repo.continuum.io/pkgs/free/, HTTP Error 403: Forbidden, DNS finished in 0.00019788742065429688 sec. ----------Environment---------- KMP_DUPLICATE_LIB_OK="True" KMP_INIT_AT_FORK="FALSE"

apache / mxnet