Hardware accelarated video playback in upstream Chromium

[matt-oakes]

What would be involved in upstreaming the patch to allow hardware accelarated video playback in standard Chromium builds and avoid the need for that part of the patch?

Currently builds of Electron, which uses Chromium internally, do not provide hardware accelarated video playback on Raspberry Pi's. There was a discussion about applying the patch in this repository to their Chromium build, however, they have decided they don't want to maintain the patch as it's not Electron specific. The discussion and comment from te Electron maintainer is here:

https://github.com/electron/electron/issues/34825#issuecomment-1248717747

The best solution would be to have upstream Chromium support hardware video accelaration on the Raspberry Pi. What would be involved with upstreaming this patch and is it a discussion that has been had with the Chromium maintainers already?

Thanks for all your work on this!

[matt-oakes]

Additionally, I have started a discussion about this on the Chromium media-dev mailing list to ask for their opinion on this:

https://groups.google.com/u/1/a/chromium.org/g/media-dev/c/GmIFotb7PFM

[thomaskvnze]

Hey, we are looking into this subject too. Have you written an email to ted meyer mentioned in the google group and got any feedback?

As far as I understood, the process looks like this:

2. You find a reviewer who can also push the patch into the release
3. You submit a review request here: https://chromium-review.googlesource.com/q/status:open+-is:wip
4. You pass the review process
5. It get's released.

[matt-oakes]

I haven't, no.

The issue I have is that while I know what patch is applied to chromium for the Raspberry Pi version, I don't know exactly what bits relate to hardware accelaration and which relate to other changes. It's also likely that the patch cannot be applied directly as it seems to just remove parts of the code which don't apply to the Raspberry Pi.

Maybe @XECDesign would be able to provide some guidance on this?

[XECDesign]

Nope sorry, I won't be of any help here. I just apply the provided patches and build the package. I'm not sure whether the patches in their current form are possible to upstream.

For the moment, upstreaming effort is focused on Firefox.

[matt-oakes]

Thanks for the reply @XECDesign. Do you know who wrote the original patches for Chromium or who might be able to explain what parts are consequential? The git history only shows you as being the author, but I suspect it's been lost in the file renaming over the years.

[XECDesign]

I'm a bit hesitant to volunteer somebody for unsolicited emails. If it's something they can help with, I'm sure they'll reply here or on the linked mailing list.

I suspect it would be quite a write-up to explain what's required and what isn't and under which circumstances.

[matt-oakes]

Sure, that's understandable. If you know someone personally who might be able to help then maybe you could reach out privately to them with a reference to this discussion so they're aware of it.

If that's not possible, then it's no problem at all. Like you say the right person might find these threads at some point on their own.

Again, thanks for your help with this.

[thomaskvnze]

I looked into the patch and did further research. Unfortunately, I have found a statement in the google forums which mentions that they are definitely not going to support linux on arm systems other than their own chromeos branch. Here is the statement in the thread https://issues.chromium.org/issues/41160483#comment66:

"Our goal is to have a Stable and secure browser first, and a GPU-accelerated one second, when possible. As we found out time and again, any sort of GPU acceleration has a lot of maintenance associated with it, between the multitude of configurations our users run, the general lack of quality of drivers (in particular on Linux), and the constant stream of incoming issue due to new hardware, driver, or distribution release. Many of these issues are not even directly attributable to this or that acceleration feature (e.g. causing random memory corruption or GPU hangs), so even investigating to narrow down a blacklist entry is significant non-trivial work.

As we do not have the resources to commit to dealing with this continued incoming stream of issues, and we don't want to compromise the first goal (stable and secure browser), our choice is not to enable these acceleration features on Linux."

This definitely matches with the patch we see here. The code from google is specifically written around chromeos based systems. The patch kind of tries to do the same just for the rpi. So I don't believe that there will be any chance that they are going to open this topic and start to maintain it for arm based systems.

[XECDesign]

That's partially why we had to shift to firefox. When an open source project is just a base for a commercial project, these things are bound to happen.

[thomaskvnze]

I'm not familiar with low level development and this topic at all. From a high level view, the patch seems to fix or implement the vp8 and h264 stateless codec usage of v4l2.

I have now found that something is currently in dev regarding the stateless codec in the chromium repo: https://chromium.googlesource.com/chromium/src/+/refs/heads/main/media/gpu/v4l2/stateless/

The latest submit was just 2 days ago.

Looking at this google group chat, I guess other people also try to implement these decoders for their boards: https://groups.google.com/a/chromium.org/g/chromium-dev/c/AOPuSQVz9Gs/m/Of6BviU0BQAJ