部署pytorch格式模型时，推理结果与预期不一致

[MrZ13]

平台(如果交叉编译请再附上交叉编译目标平台):

Linux admin-NUC8 5.19.0-41-generic #42~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Tue Apr 18 17:40:00 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

Github版本:

v2.5.0 https://github.com/alibaba/MNN/releases/tag/2.5.0

编译方式:

cmake -D CMAKE_BUILD_TYPE=Release \ -D MNN_OPENMP=ON \ -D MNN_USE_THREAD_POOL=OFF \ -D MNN_BUILD_QUANTOOLS=ON \ -D MNN_BUILD_CONVERTER=ON \ -D MNN_BUILD_DEMO=ON \ -D MNN_BUILD_BENCHMARK=ON ..

目标：

在Linux系统的NUC8上部署pytorch训练的模型

问题描述：

• 按照文档中的做法，将pytorch模型转为onnx格式，再转为mnn格式，并且使用testMNNFromONNX.py脚本进行了正确性校验，显示TEST_SUCCESS;
• 参考demo/exec/multiPose.cpp和文档，写了推理模型的代码，但是发现推理结果和pytorch格式的模型不一致；
• 使用像素值全为1或全为0~255之间的某个特定值的图片进行测试，推理结果无误，和pytorch格式的模型一致；
• 考虑可能仍然是数据排布相关的问题，但是我不太清楚具体要如何修改代码，因此想请教一下，十分感谢

以下是完整的推理代码：

include

include

include

include

define STB_IMAGE_IMPLEMENTATION

define STB_IMAGE_WRITE_IMPLEMENTATION

include "stb_image.h"

include "stb_image_write.h"

include <MNN/expr/ExprCreator.hpp>

include "../tools/cv/include/cv/cv.hpp"

include <MNN/ImageProcess.hpp>

include <MNN/Interpreter.hpp>

define MNN_OPEN_TIME_TRACE

include <MNN/AutoTime.hpp>

using namespace MNN; using namespace MNN::CV;

int main(int argc, char* argv[]) { if (argc < 3) { std::cout << "Usage: ./derDetection.out derDetection.mnn input.jpg" << std::endl; }

const auto derModel           = argv[1];
const auto inputImageFileName  = argv[2];

int width, height, channel;
auto inputImage = stbi_load(inputImageFileName, &width, &height, &channel, 0);

if (nullptr == inputImage) {
    MNN_ERROR("Can't open %s\n", inputImage);
    return 0;
}
MNN_PRINT("origin size: %d, %d, %d\n", width, height, channel);

auto mnnNet = std::shared_ptr<MNN::Interpreter>(MNN::Interpreter::createFromFile(derModel));
MNN::ScheduleConfig netConfig;
netConfig.type      = MNN_FORWARD_CPU;
netConfig.numThread = 4;

auto session = mnnNet->createSession(netConfig);
auto input = mnnNet->getSessionInput(session, nullptr);

// preprocess input image
{
    Matrix trans;
    trans.setIdentity();

    // 图像缩放为{1, 3, 72, 72}的尺寸
    trans.postScale(1.0f / 72.0f, 1.0f / 72.0f);
    trans.postScale(width, height);

    ImageProcess::Config preProcessConfig;
    preProcessConfig.sourceFormat = CV::RGB;
    preProcessConfig.destFormat   = CV::RGB;
    preProcessConfig.filterType   = CV::BILINEAR;

    std::shared_ptr<ImageProcess> pretreat(ImageProcess::create(preProcessConfig));
    pretreat->setMatrix(trans);

    pretreat->convert(inputImage, width, height, 0, input);
    input->printShape();

}
// inference begin
{
    AUTOTIME;
    mnnNet->runSession(session);
}

auto outputTensor = mnnNet->getSessionOutput(session, NULL);
auto nchwTensor = new Tensor(outputTensor, Tensor::CAFFE);
outputTensor->copyToHostTensor(nchwTensor);

auto firstScore = nchwTensor->host<float>()[0];
auto secondScore = nchwTensor->host<float>()[1];
auto thirdScore = nchwTensor->host<float>()[2];
auto fourthScore = nchwTensor->host<float>()[3];
auto fifthScore = nchwTensor->host<float>()[4];
auto sixthScore = nchwTensor->host<float>()[5];
MNN_PRINT("value of categories: %f, %f, %f, %f, %f, %f\n", firstScore, secondScore,
          thirdScore, fourthScore, fifthScore, sixthScore);

// size返回的结果是tensor的字节数
auto size = nchwTensor->size();
auto tensorWidth = nchwTensor->width();
auto tensorHeight = nchwTensor->height();
auto tensorChannel = nchwTensor->channel();
MNN_PRINT("value of size, width, height and channel: %d, %d, %d, %d\n", size, tensorWidth,
          tensorHeight, tensorChannel);

delete nchwTensor;
stbi_image_free(inputImage);
return 0;

}

[wangzhaode]

模型转换的时候加上参数--keepInputFormat试一下

[wangzhaode]

直接将数据写到input中可能会有格式问题，正常来说需要写到一个tmpInput Tensor中，然后使用input->copyFromHost(tmpInput); 才可以保证没有格式问题

[MrZ13]

直接将数据写到input中可能会有格式问题，正常来说需要写到一个tmpInput Tensor中，然后使用input->copyFromHost(tmpInput); 才可以保证没有格式问题

感谢回复，请问写入temp tensor的时候是否就需要按照nchw的排布方式调整数据？

[github-actions[bot]]

Marking as stale. No activity in 60 days.