search

microsoft / FFmpegInterop

This is a code sample to make it easier to use FFmpeg in Windows applications.
Apache License 2.0
1.31k stars 310 forks source link

Hardware accelerated decoding #183

Open lukasf opened 7 years ago

lukasf commented 7 years ago

Is it possible to enable hardware acceletation in FFmpegInterop? I see in the config script that d3d11 and dxva are explicitly disabled. Why is that?

Having hardware acceleration enabled would greatly increase the usability of this lib. It would allow decoding of HEVC, H264, VC1, VP9 (depending on CPU type) even on medicore devices, and also for other formats such as MPEG2 it would drastically reduce power consumption and noise level during decoding.

I know that at least for H264, it is possible to pass through (force video decode = false), which enables HW acceleration down the decoder chain if possible. But it does not work with Hi10P profile. And HEVC, MPEG2, VC1, VP9 etc are all not supported even if the IGP/GPU would support HW acceleration. It would be awesome if we could use ffmpeg internal HW acceleration instead of having to rely on passthroug solutions (which anyways is only possible for codecs officially supported by WinRT EncodingParameter APIs).

brabebhin commented 7 years ago

Last time I checked these, it had some issues with store certification. But it may not be the case anymore. I actually tried compiling ffmpeg for windows store some time before this project was born. There quite a few issues with store cert, even if dlls did work. I could never get the entire "disabled" list checked out.

I also think keeping the backwards compatibility with windows phone 8.1 is at play. For example, we do not use hardware accelerated flac decoding either, even if this is supported in winRT in windows 10.

lukasf commented 7 years ago

Windows Phone is officially dead now, no dev would do any new development for that platform anymore. So unfortunately, this should not be a concern. I really loved WP/W10M, but Nadella hated it and he destroyed it like a maniac. I really don't know what that guy was thinking... Sad story.

API compatibility might be an issue.

lukasf commented 7 years ago

Oh man, this is pretty disappointing: It looks like hardware acceleration is not really implemented in the ffmpeg library. There is some groundwork done, but the important decoder methods such as init, uninit, create buffer, get results, etc are all not available in the lib. Even if you enable hardware acceleration in the build (which did not introduce API warnings, by the way), the default lib implementation will skip all hardware accelerated decoders. And if you force it to use one, it won't work because all the methods are not there.

The real implementations for hardware decoders such as dxva2 are only available in the ffmpeg.exe project. And they are written in a way so they can only work in the context of the ffmpeg.exe application, because they heavily use all its internal stuff. It would probably require heavy rework to make these work in FFmpegInterop.

That's pretty poor design on ffmpeg side if you ask me. Why are hardware decoders not part of the lib, just like software decoders? Oh man...

brabebhin commented 7 years ago

That is somewhat expected, because not all platforms running FFMPEG are capable of hardware decoding. My guess is that they didn't bother with integrating it into the library because the library is already pretty huge as it stands.

lukasf commented 7 years ago

Since every hardware decoder must be explicitly enabled in the config, Linux users would not have any downsides if a "dxva-enabled" build contains more than just a few strings and basic definitions...

payyavula commented 4 years ago

@lukasf and @mcosmin222 I am using FFmpeg 2.3 and FFmpegInteropX to decode the RTP stream on HoloLens-2. It works fine for the H264 video stream, but when I enable, H265, FFmpeg crashes. checking, if H265 hardware acceleration is supported configuration on HoloLens-2?