mxnet --> onnx converted insightface arcface model generates incorrect inference results.

I am trying to convert the Insightface arcface LResNet100E-IR,ArcFace@ms1m-refine-v2 mxnet model to work with onnx.

Based on this issue, it looks like mxnet only supports up to onnx v1.3.0. I am therefore using the following library verions:

mxnet==1.7.0.post1
onnx==1.3.0
onnxruntime==1.6.0

It looks like out of the box the arcface model is not support by onnx, but I came across this issue which which links a script for properly converting the arcface model to onnx.

Running the script converts the model from mxnet format to onnx format successfully, however the output is no longer correct. Here is my mxnet script I use for running inference (using a pre-aligned face chip):

#############################
# Inference with mxnet
#############################
import numpy as np
import cv2
import mxnet as mx
from collections import namedtuple

import pkg_resources
print("mxnet version:", pkg_resources.get_distribution("mxnet").version)

Batch = namedtuple('Batch', ['data'])
def getEmbedding(face_chip):
    face_chip = np.swapaxes(face_chip, 0, 2)
    face_chip = np.swapaxes(face_chip, 1, 2)
    face_chip = face_chip[np.newaxis, :]
    array = mx.nd.array(face_chip)
    mod.forward(Batch([array]))
    out = mod.get_outputs()[0].asnumpy()
    return out

prefix = "./models/insightface/model"
sym, arg, aux = mx.model.load_checkpoint(prefix, 0)
ctx = mx.cpu()
mod = mx.mod.Module(symbol=sym, context=ctx, label_names=None)
mod.bind(for_training=False, data_shapes=[('data', (1,3,112,112))])
mod.set_params(arg, aux)

img = cv2.imread("face_chip.jpg")

img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)

res = getEmbedding(img)
print(res[0][0])

I print the first element of the feature vector, which is: 2.216425

Now here is my script for running inference with onnxruntime:

#############################
# Inference with onnx runtime
#############################
import json
import numpy as np
import cv2
import onnx
import onnxruntime
from onnx import numpy_helper

import pkg_resources
print("onnx-runtime version:", pkg_resources.get_distribution("onnxruntime").version)
print("onnx version:", pkg_resources.get_distribution("onnx").version)

model="./models/insightface/arcface_r100.onnx"

#Preprocess the image
img = cv2.imread("face_chip.jpg")

img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
img.resize((1, 3, 112, 112))
data = json.dumps({'data': img.tolist()})
data = np.array(json.loads(data)['data']).astype('float32')

session = onnxruntime.InferenceSession(model, None)
input_name = session.get_inputs()[0].name
output_name = session.get_outputs()[0].name

result = session.run([output_name], {input_name: data})
print(result[0][0][0])

Again, I print the first element of the feature vector, which is now: -0.19618489.

This is clearly not correct. What have I done wrong? Has the conversion failed?

linghu8812 / tensorrt_inference

mxnet --> onnx converted insightface arcface model generates incorrect inference results. #57