natmlx / natml-unity

High performance, cross-platform machine learning for Unity Engine.
Apache License 2.0
228 stars 25 forks source link

Pixel buffers from `MLVideoFeature` have zero alpha #26

Closed Joakim432710 closed 2 years ago

Joakim432710 commented 2 years ago

Reproducible on Windows Unity Editor

var tex = new Texture2D(pixelData.width, pixelData.height, TextureFormat.RGBA32, false);
var reader = new MLVideoFeature(pathWithProtocol);
foreach (var frame in reader)
{
    frame.feature.CopyTo(tex);    
    #if UNITY_EDITOR_WIN
    var ptr = (byte*)tex.GetRawTextureData<byte>().GetUnsafePtr();
    for (var iii = 3; iii < tex.width * tex.height * 4; iii += 4)
        ptr[iii] = 255;
    #endif
    tex.Apply(false);
}

If you log the pixel buffers out you will see that the alpha byte is zero when decoding mp4 h264 video generated by NatCorder.

olokobayusuf commented 2 years ago

Hey @Joakim432710 , I can't reproduce this at all. What version of Windows are you on? NatML natively requests RGBX pixel buffers from Windows MediaFoundation; and RGBX is guaranteed to return pixel buffers with alpha = 255 (I can confirm this on Windows 11 21H2). Can you share the video you are testing with, or a full repro project?

olokobayusuf commented 2 years ago

Also tried to repro with a video recorded with NatCorder in the Windows editor--still can't repro.

olokobayusuf commented 2 years ago

Closing due to lack of activity and inability to reproduce.