Multiple images for batch inference

[babak-abad]

Hi, Serizba :)

I'm trying to use your library in windows 10 x64 machine in VS2019 C++. I want to inference my model for batch of images ( vector < cv::Mat > ). I write a simple code as below for single image and it works correctly:

string im_path{ "..." };
string model_path{ "...\\ocr_model" };

cv::Mat tmp, im;

cv::resize(cv::imread(im_path, cv::IMREAD_GRAYSCALE), tmp, cv::Size(127, 25), 0, 0, cv::INTER_CUBIC);

cv::transpose(tmp, im);

int rows = im.rows; int cols = im.cols; int channels = im.channels();

// Put image in tensor 
std::vector<uint8_t> img_data;

auto e = std::end(img_data);
img_data.insert(e, im.data, im.data + im.total() * channels);

auto input = cppflow::tensor(img_data, {rows, cols, channels});

input = cppflow::cast(input, TF_UINT8, TF_FLOAT);
auto t = input.get_data<float>();

input = input / 255.f;
input = cppflow::expand_dims(input, 0);

cppflow::model model{ model_path };
auto output = model({ {"serving_default_input:0", input}}, { "StatefulPartitionedCall:0"});

I want to load multiple images (in code below I use a cloned image as second image). here is what I really want to do:

string im_path{ "..." };
string model_path{ "...\\ocr_model" };

cv::Mat tmp, im;
cv::resize(cv::imread(im_path, cv::IMREAD_GRAYSCALE), tmp, cv::Size(127, 25), 0, 0, cv::INTER_CUBIC);

cv::transpose(tmp, im);

int rows = im.rows; int cols = im.cols; int channels = im.channels();

// Put image in tensor 
std::vector<uint8_t> img_data; 

auto im_clone = im.clone();
auto e = std::end(img_data);
img_data.insert(e, im.data, im.data + im.total() * channels);
e = std::end(img_data);
img_data.insert(e, im_clone.data, im_clone.data + im_clone.total() * channels);

auto input = cppflow::tensor(img_data, {2, rows, cols, channels});

input = cppflow::cast(input, TF_UINT8, TF_FLOAT);

input = input / 255.f;
input = cppflow::expand_dims(input, 0);

cppflow::model model{ model_path };
auto output = model({ {"serving_default_input:0", input}}, { "StatefulPartitionedCall:0"});

As you see the difference between the codes are img_data preparation and tensor definition but unfortunately, I get this error:

Unhandled exception at 0x00007FFFF4514ED9 in cppflow_Test.exe: Microsoft C++ exception: std::runtime_error at memory location 0x00000031F72FDBD8.

How can I load multiple images (vector< cv::Mat >) to a tensor?

[serizba]

Looks like you should not expand the dims in the second example, as it already has a batch dimension.