bytedeco / javacv

Java interface to OpenCV, FFmpeg, and more
Other
7.39k stars 1.56k forks source link

Passing OpenCV Mat as Tensorflow input #2220

Open barrypitman opened 2 months ago

barrypitman commented 2 months ago

Firstly, thanks for a great project, it's fun to work with!

I have an org.bytedeco.opencv.opencv_core.Mat object that I have prepared into the correct shape, format & scaling to pass to a tensorflow lite model. I'm having trouble populating the model input tensor correctly.

Mat image = ...; // 300 x 300 x 3 channels, depth=32
FloatPointer input = interpreter.typed_input_tensor_float(0);

I've got it working by iterating over the rows and cols of the image and copying the image pixels one by one into a float[] and calling input.put() on the float array. However, that seems inefficient, and I was hoping that I could do something like input.put(image), but that isn't working, or at least the model outputs aren't correct.

Could you give me any pointers (!) on how to efficiently copy/pass the Mat as a tensorflow input?

Thanks in advance.

saudet commented 2 months ago

Indexers are efficient: https://bytedeco.org/news/2014/12/23/third-release/

barrypitman commented 2 months ago

So something like this? input.put((float[]) image.createIndexer(false).array());

I was wondering whether it would be better to pass a pointer to the Mat (and avoid loading the floats into java memory at all), but that didn't seem to work.