Buffer Pool and ToBytes / ImEncode

[turowicz]

Is it possible to make use of a buffer pooling technique for converting Mats to byte[]? I'm processing a large amount of video frames and feed it to TensorFlow. Currently I have this ugly intermediate step that makes the conversion so that TensorFlow can create an input object.

[turowicz]

cc @shimat maybe you know what's the best approach to this

[shimat]

I don't have a good answer, but one thing we can do is use and keep one vector to encode. https://github.com/shimat/opencvsharp/blob/master/src/OpenCvSharp/Cv2/Cv2_imgcodecs.cs#L193

To do this, it is needed to wrap std::vector.clear().

using var buffer = new VectorOfByte();

while (true)
{
    using Mat frame = GetFrame();

    NativeMethods.imgcodecs_imencode_vector(".png", frame.CvPtr, buffer.CvPtr, null, 0);

    byte[] encodedBytes = buffer.ToArray();

    buffer.Clear(); // not implemented
}

[turowicz]

buffer.ToArray() won't this allocate a new array though?

[shimat]

Ah ... VectorOfT.ToArray allocates memory each time, so we have to use ArrayPool. https://github.com/shimat/opencvsharp/blob/master/src/OpenCvSharp/Vector/VectorOfByte.cs#L90

This is a pseudocode:

var pool = System.Buffers.ArrayPool<byte>.Shared;
using var bufferVec = new VectorOfByte();

using Mat frame = GetFrame();

// Call cv::imencode
NativeMethods.imgcodecs_imencode_vector(".png", frame.CvPtr, bufferVec.CvPtr, null, 0);

// vector<uchar>  =>  byte[]
byte[] bufferArray = pool.Rent(bufferVec.Size);
try
{
    Marshal.Copy(bufferVec.ElemPtr, bufferArray, 0, bufferArray.Length);

    // Use bufferArray 
}
finally
{
    pool.Return(bufferArray);
}

[turowicz]

Thank you @shimat !