Incorrect rendering h265/HEVC video with alpha channel

[araid]

Most appropriate sub-area of p5.js?

• [ ] Accessibility
• [ ] Color
• [X] Core/Environment/Rendering
• [ ] Data
• [ ] DOM
• [ ] Events
• [X] Image
• [ ] IO
• [ ] Math
• [ ] Typography
• [ ] Utilities
• [ ] WebGL
• [ ] Build Process
• [ ] Unit Testing
• [ ] Internalization
• [ ] Friendly Errors
• [ ] Other (specify if possible)

p5.js version

1.6.0

Web browser and version

111.0.5563.64 (Official Build) (arm64)

Operating System

macOS Ventura 13.2.1

Steps to reproduce this

When using image() to render a video with an alpha channel, pixels from the previous frames are not being cleared.

Steps:

1. Export a video with alpha channel using h265 codec (macOS can convert to by right click > encode selected video files)
2. Load it in a p5.js script with createVideo()
3. Display it with image()
4. Observe frames not clearing.

Snippet:

• Example of bug when rendering inside canvas: https://editor.p5js.org/araid/sketches/TcKkF4gOb
• The video renders correctly outside canvas (not using image()): https://editor.p5js.org/araid/sketches/7beM-GM1k

[welcome[bot]]

Welcome! 👋 Thanks for opening your first issue here! And to ensure the community is able to respond to your issue, please make sure to fill out the inputs in the issue forms. Thank you!

[davepagurek]

I think the fix for this would be to do a clearRect before drawing the media element to its canvas here: https://github.com/processing/p5.js/blob/383efd0/src/dom/dom.js#L3161

That said, h.265 (or transparent vp9) won't be expected to fully work cross-browser, since Firefox doesn't yet support h.265 transparency, and Safari doesn't yet support vp9 transparency. I've also noticed issues with 10-bit color encoding in h.265 resulting in differing outputs in different browsers too. So while this particular issue would be great to fix, it'll probably still be a while before we want to officially support this :')

[limzykenneth]

Firefox has already indicated that they will not be supporting h265 because of the patents issue, preferring to support AV1. AV1 currently have no alpha support. The options left are either choose h265 or vp9 with alpha (and not support either Firefox or Safari) or do some kind of feature check that fallback to either h265 or vp9 depending on which is supported.

[araid]

Thanks for the context, I wasn't aware of the different support levels across browsers.

@davepagurek I tested your suggestion but it didn't fix the issue: the previous frames are still displayed, even if I call clearRect() or even fillRect() before the drawImage() in the the line you linked to.

After more testing I realized that the issue doesn't happen in Safari, so I'm wondering if it might just be a bug in Chrome.

[davepagurek]

That is very weird! It looks like the clearRect is necessary to fix Safari, but Chrome still shows trails. It does indeed sound like a Chrome bug (maybe somewhere in the Chrome source, they need an equivalent of a clearRect somewhere too 😅)