Hardware accelration

[DemiMarie]

On Fedora, OpenH264 is the only way to support H.264 decoding. Many GPUs have support for accelerating this decoding, and it would be nice if OpenH264 could use them.

[Darkspirit]

In other words: The Firefox binary can't contain full ffmpeg (https://bugzilla.mozilla.org/show_bug.cgi?id=1737116) and the Firefox Flatpak package can't have a hard dependency on org.freedesktop.Platform.ffmpeg-full because https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/vaapi_h264.c contains patented code (Decoded Picture Buffer). Could the OpenH264 binary somehow contain ffmpeg's vaapi_h264.c?

[huili2]

Not yet. Till now openh264 is a pure independent software-based codec, and has no plan to support either hardware acceleration or other codecs.

[DemiMarie]

Could it at least contain that particular part as a library function?

[huili2]

Basically it would not contain any non-system libraries in openh264, which will break the easy integration of it. But as an addition, it may be an alternation if other contributors could help on it :)

[DemiMarie]

Basically it would not contain any non-system libraries in openh264, which will break the easy integration of it.

One option would be to dynamically load these libraries at runtime.

But as an addition, it may be an alternation if other contributors could help on it :)

@Darkspirit interested?

[huili2]

will close this if no response from owner.

[Darkspirit]

interested?

no, sorry

[noloader]

@huili2,

... openh264 is a pure independent software-based codec, and has no plan to support either hardware acceleration or other codecs.

openH264 has hardware acceleration. Single instruction multiple data (SIMD) instructions are hardware acceleration. SSE, NEON and ASIMD are examples of hardware acceleration.

• https://github.com/cisco/openh264/tree/master/codec/encoder/core
• https://github.com/cisco/openh264/tree/master/codec/decoder/core

What openH264 seems to lack is GPU acceleration. That is a different type of hardware acceleration. Lack of GPU support does not mean openH264 does not have hardware acceleration.

(Sorry to split hairs. This has come up on another forum, and terms are being muddled).

[DemiMarie]

@huili2,

... openh264 is a pure independent software-based codec, and has no plan to support either hardware acceleration or other codecs.

openH264 has hardware acceleration. Single instruction multiple data (SIMD) instructions are hardware acceleration. SSE, NEON and ASIMD are examples of hardware acceleration.

• https://github.com/cisco/openh264/tree/master/codec/encoder/core
• https://github.com/cisco/openh264/tree/master/codec/decoder/core

What openH264 seems to lack is GPU acceleration. That is a different type of hardware acceleration. Lack of GPU support does not mean openH264 does not have hardware acceleration.

When it comes to codecs, “hardware acceleration” usually refers to dedicated silicon that exists solely for the purpose of media encoding and decoding.

[DemiMarie]

Basically it would not contain any non-system libraries in openh264, which will break the easy integration of it. But as an addition, it may be an alternation if other contributors could help on it :)

Any non-system dependencies could be provided by the caller as function pointers.

[noloader]

When it comes to codecs, “hardware acceleration” usually refers to dedicated silicon that exists solely for the purpose of media encoding and decoding.

MMX is Multimedia Extensions. Back when MMX was introduced, that was one of the applications of the SIMD unit. Applications were not limited to multimedia streams. Scientific applications could also use the SIMD unit.

The progression of MMX to SSE, or SSE to AVX, or NEON or ASIMD have not changed anything. It is still dedicated hardware that can be used for multimedia streams (among other things).

GPU acceleration is a specialization of hardware acceleration. GPU's are used for scientific programs, too. I doubt you will find a commodity GPU that solely operates on multimedia.

I don't like citing wikipedia, but take a look at the terms used at https://en.wikipedia.org/wiki/Graphics_processing_unit#GPU_accelerated_video_decoding_and_encoding.

[garybuhrmaster]

At least partially related to #1841, #3270, and #3127.

They (and this issue) should probably all be closed as wontfix (as was suggested in the 6th reply), as no one has been willing to resource the changes, let alone having their organizations IP lawyers review whether it is acceptable to even consider doing so (the legal review is likely going to be the hard part).

[DemiMarie]

At least partially related to #1841, #3270, and #3127.

They (and this issue) should probably all be closed as wontfix (as was suggested in the 6th reply), as no one has been willing to resource the changes, let alone having their organizations IP lawyers review whether it is acceptable to even consider doing so (the legal review is likely going to be the hard part).

Could Cisco itself do it?

[huili2]

yeah. Agree that in video codec, hw acceleration usually indicates GPU acceleration. And there's no plan to support it. So close all above related issues.

[sgf]

yeah. Agree that in video codec, hw acceleration usually indicates GPU acceleration. And there's no plan to support it. So close all above related issues.

GPU acceleration is divided into general acceleration and native acceleration.

Native acceleration requires chip manufacturers to develop codecs themselves, which are on the same level and in competition with OpenH264.

General acceleration requires the use of chip-specific technologies such as CUDA/OpenCL.

In short, supporting GPU acceleration is a labor-intensive task.

[DemiMarie]

The reason this issue exists is that on many Linux distributions, OpenH264 is the only allowed H.264 coded, due to patent reasons. As such, even a solution that is technically disgusting would be acceptable to me, such as vendoring a copy of Mesa, intel-media-driver, ffmpeg, or any combination. The purpose is not to provide a technically sound solution, but rather to provide a way for distributions to ship accelerated H.264 support without infringing on any patents and without paying royalties themselves. Extending the licensing program used for OpenH264 to Mesa would work just as well.