Conversion from OpenCV Mat to TorchSharp Tensor (C, H, W) format.

[TheBlackPlague]

Summary of your issue

Is it possible to convert OpenCV's Mat object to a TorchSharp Tensor? If I'm correct, the Mat is typically in the format: (W, H, C). My goal is to convert it into a TorchSharp Tensor in the format: (C, H, W).

I imagine this could be possible in the following steps:

Mat image; // An image
image.GetArray(out byte[] data);
Tensor dataTensor = from_array(array: data, type: ScalarType.Byte, device: "cuda");
Tensor imageTensor = dataTensor.reshape(image.Width, image.Height, 3).permute(2, 1, 0); // Required tensor

If I'm correct in the assumptions + pseudo-code I wrote above, please let me know. And if so, is it possible to add a method for this conversion directly into OpenCV? Many people, like myself, may intend to use OpenCV with TorchSharp.

[TheBlackPlague]

I figured it out:

Mat image; // An image
image.ToArray(out byte[,,] data);
Tensor imageTensor = from_array(data, ScalarType.Byte, "cuda").permute(2, 0, 1);

void ToArray(this Mat image, out byte[,,] data)
{
    byte[,,] dataTemp = new byte[image.Height, image.Width, 3];

    Parallel.For(0, image.Width, w =>
    {
        for (int h = 0; h < image.Height; h++) {
            Vec3b vec = image.At<Vec3b>(h, w);
            dataTemp[h, w, 0] = vec.Item0;
            dataTemp[h, w, 1] = vec.Item1;
            dataTemp[h, w, 2] = vec.Item2;
        }
    });

    data = dataTemp;
}

I've pasted the above code snippet for anyone who may find it helpful.