Python prediction and frugally prediciton are similar but not same

[furkank75]

Hello,

I implement the frugally Code, but I see that the prediction are not the same. I thing my types of the images are not right..

Prediction in C++:

const auto input1 = fdeep::tensor5_from_bytes(image1.ptr(),
        static_cast<std::size_t>(image1.rows),
        static_cast<std::size_t>(image1.cols),
        static_cast<std::size_t>(image1.channels()),
        0.0f, 255.0f);

const auto input1 = fdeep::tensor5_from_bytes(image1.ptr(),
        static_cast<std::size_t>(image1.rows),
        static_cast<std::size_t>(image1.cols),
        static_cast<std::size_t>(image1.channels()),
        0.0f, 1.0f);

Prediction in Python:

My jsonFile / Code / Test Images: Neuer Ordner (2).zip

Is there a Type Error? How can I solve this? Thanks a lot..

[Dobiasd]

Hi Furkan,

the fact that the automatic test, which runs in the load_model succeeds, confirms that the model is working correctly in C++, i.e., it produces the same output as in Python.

This points to something with the image format (or pixel-value scaling) being wrong, yes.

Can you provide a minimal example in Python too, so I can run and compare both versions?

[furkank75]

I check the pixel values of input images in C++ and python.. This is the same..

I add the Python Code in few miniutes.

[furkank75]

TEST_frugally.zip minimal example in python..

[Dobiasd]

In the C++ version, you are creating the target image with

const cv::Mat image2(cv::Size(32, 32), 1);

The 1 is the same as CV_8S, which is not what you want. It should be CV_8UC1 instead.

And the C++ version is lacking the thresholding at 0.45.

I fixed both, made the minimal examples a bit more similar, and removed some unneeded stuff. Now the output from C++ is the same as from Python.

# fdeep_issue_197.py

import cv2
import numpy as np
from keras.models import load_model

model = load_model("model.h5")

image = cv2.imread("image.png", 0)
image1 = cv2.imread("image.png", 0)

input1 = np.asarray(image)
input1 = np.reshape(input1, (1, 32, 32, 1))

input2 = np.asarray(image1)
input2 = np.reshape(input2, (1, 32, 32, 1))

pred = model.predict([input1, input2], verbose=1, batch_size=1)
pred_t = pred > 0.45

pred_out = pred_t.reshape(32, 32)
pred_out = pred_out.astype(np.float32) * 255

cv2.imwrite("result.python.png", pred_out)

// fdeep_issue_197.cpp

#include <fdeep/fdeep.hpp>
#include <opencv2/opencv.hpp>

int main()
{
    cv::Mat image  = cv::imread("image.png", 0);
    cv::Mat image1 = cv::imread("image.png", 0);

    assert(image.isContinuous());
    assert(image1.isContinuous());

    const auto model = fdeep::load_model("model.json");

    const auto input = fdeep::tensor5_from_bytes(image.ptr(),
        static_cast<std::size_t>(image.rows),
        static_cast<std::size_t>(image.cols),
        static_cast<std::size_t>(image.channels()),
        0.0f, 255.0f);

    const auto input1 = fdeep::tensor5_from_bytes(image1.ptr(),
        static_cast<std::size_t>(image1.rows),
        static_cast<std::size_t>(image1.cols),
        static_cast<std::size_t>(image1.channels()),
        0.0f, 255.0f);

    const auto result = model.predict({ input, input1 });

    cv::Mat image2(cv::Size(32, 32), CV_8UC1);
    fdeep::tensor5_into_bytes(result[0], image2.data, image2.rows * image2.cols * image2.channels());
    cv::threshold(image2, image2, static_cast<int>(0.45 * 255), 255, cv::THRESH_BINARY);

    cv::imwrite("result.cpp.png", image2);
}

Btw., you provided CropImage2.png and CropImage2_1.png, but only used CropImage2.png, so I did the same when testing (only renamed it to image.png).

python3 fdeep_issue_197.py
g++ -std=c++14 -O3 fdeep_issue_197.cpp `pkg-config --libs opencv` -o fdeep_issue_197
./fdeep_issue_197

Hope that helps. :slightly_smiling_face:

[furkank75]

Nice, thank you very much..