video issues on windows target

[wighawag]

Video texture has some issues on Windows 10 using direct3d9

The video tested can be downloaded from there : https://drive.google.com/drive/folders/0B6lfi9ljiCTuU2VzdXBlZ2FFdWc

its name is : "8K Sample Video Park_1.mp4"

on Kha windows it crash at runtime with a read access violation on the line :

affirm(texture->LockRect(0, &rect, 0, 0));

in TextureImpl.cpp in u8* Graphics4::Texture::lock() when using direct3d9

on Kha html5, khamake get stuck converting the webm version but it seems to have done its job after a while. Upon relaunching khamake, khamake finish. the html5 version seem to work with that 8k version but loading it seems to not always work as loading take forever sometimes.

This file actually failed to be played on all the tested player on my system. Some froze, some play for a while (Unity) and some said they cannot play it (windows player).

By scaling it down by 2 using

ffmpeg.exe -i "8K Sample Video Park_1.mp4" -vf scale=3840:-1 "4K Sample Video Park_1.mp4"

We get a 4K version and this one work on all tested players including unity.

On Kha windows it does not crash anymore but the result is very slow and chopy and the colors are not correct. I played the avi file and it looks correct.

on Kha html5 it works fine though

Also I noticed the quality of the video in the resulting avi file is horrible. Why can't we use the mp4 as it ?

I also noticed a difference between html5 and windows target: on html5 even if the video.play() method is not called, the video texture contain something (I guess the first frame). On Windows it is black. Not a big issue though

[wighawag]

created a test project here with the 4k video : https://github.com/wighawag/kha_video_test

[RobDangerous]

Video decoding is mostly handled by the target system itself. In case of D3D9 that's DirectShow and I would suspect that won't be able to handle 8k videos. D3D11 currently uses DirectShow as well but I'll eventually change that to use Media Foundation which will probably help. For browsers mp4 can't work everywhere (because patents) so we need a webm version. Could change khamake though to just copy the original mp4 for browsers which support it. Also the video export parameters in khamake need tweaking in general (maybe you could have a look yourself).

[wighawag]

Fair enough for 8k videos but the 4k video is being played without problem in a video player that use DirectShow (set in the settings).

in Kha the color are also off, like the hue is off by some amount. It plays very slowly too, like it freeze every half second. I could try to look into it if I can get some pointer.

If you could try the sample and see if you get the same issue it would be great.

As for Media Foundation, any timeline ? My project would need to decide what we are going to use and would love to use Kha :)

as for conversion to avi, I was actually referring to the windows target. DirectShow should support mp4 and since I got a mp4 as input, it would not need to convert it it avi. I ll try that if it works

Thanks

[RobDangerous]

Looked into it and can start implementing it after the Haxe Summit - so, end of September should be realistic.

[wighawag]

Cool that sounds good

[RobDangerous]

Looked into it some more and it turns out more complicated than I thought. Best reference seems to be https://stackoverflow.com/questions/19846770/how-do-i-use-hardware-accelerated-video-h-264-decoding-with-directx-11-and-windo I think I need help.

[wighawag]

Ok, what did you gather so far? What is missing ? a h264 parser? in the follow up comment he seems to have solve his original issue but that support less format. Could it be a first step for Kha? He did not provide much info though. Is that something you tried?

[RobDangerous]

Didn't try it, I'm scared by that link and will try to give it to my HiWi. Hoped it would be a one weekend job but looks considerably more complicated due to the format parsing things. No idea about that later reply, that's not very informative.

[RobDangerous]

Oh, I found sample code, maybe that'll help: https://github.com/Microsoft/Windows-classic-samples/tree/master/Samples/DX11VideoRenderer

[RobDangerous]

So this creates an IMFMediaSink which I can use for I don't know. Video decoding APIs are always such an IMFTimeSink. Any help is really appreciated here, if somebody could make that DX11VideoRenderer actually play a video somehow, that would be great..

[wighawag]

I dig a bit deeper in the windows api mess but could not figure out much. Their sample are not really self explanatory.

These links refer to dx11 : https://msdn.microsoft.com/en-us/library/windows/desktop/hh162912

these are for dx9 : using direct show : https://msdn.microsoft.com/en-us/library/windows/desktop/aa965245(v=vs.85).aspx using media fundation : https://msdn.microsoft.com/en-us/library/windows/desktop/aa965266(v=vs.85).aspx

and some info on DXVA-HD : https://msdn.microsoft.com/en-us/library/windows/desktop/ee663581(v=vs.85).aspx

There is also the source code of the "MPC-HC" video player (use dx9) for inspiration : https://github.com/mpc-hc/mpc-hc/blob/develop/src/filters/renderer/VideoRenderers/DX9RenderingEngine.cpp

Unfortunately, I have still to learn directx or any windows api. I ll see if I can get deeper again but thought these links my help

[RobDangerous]

@MarianThull is on it, but it turned out very complicated, so no guarantees it'll work out.