search

bbc / VideoContext

An experimental HTML5 & WebGL video composition and rendering API.
http://bbc.github.io/VideoContext/
Apache License 2.0
1.33k stars 157 forks source link

⚡️ Performance improvement when rendering ImageNode #103

Closed germain-gg closed 5 years ago

germain-gg commented 6 years ago

Hi,

First of all, thank you very much for VideoContext. It's great!

I have noticed some poor performances on "low-end" or older laptop when rendering an ImageNode. Running a performance audit in the devtools would show something like:

screen shot 2018-09-10 at 21 01 44

The ImageNode starts playing when the "green" bit gets bigger. In the Chrome devtools the "green" part of the chart refers to a painting operation.

If we have a closer look at the call stack, we can see an "Image Decode" event that takes place everytime that gl.texImage2D is called.

screen shot 2018-09-10 at 21 02 00

After digging a bit more on gl.texImage2D I noticed that we can draw on a texture using an ImageBitmap as a source.

The ImageBitmap interface represents a bitmap image which can be drawn to a without undue latency. It can be created from a variety of source objects using the createImageBitmap() factory method. ImageBitmap provides an asynchronous and resource efficient pathway to prepare textures for rendering in WebGL.

I have then changed the ImageNode class a bit, and after running some tests the timeline does less painting operations as the "Image Decode" step is completely gone from the call stack.

screen shot 2018-09-10 at 20 58 57

It looks like this new Image source has some sporadic support. I have decided to gracefully degrade to use HTMLImageElement if the user agent does not support ImageBitmap.

Keen to hear your thoughts on that, I'd be happy to answer your questions if you have any 👍

PTaylour commented 6 years ago

Fantastic! This is great :)

PTaylour commented 6 years ago

@gsouquet do you have allow edits from maintainers checked for your pull request?

I've got a couple of regression tests to add that cover the image node with and without createImageBitmap

germain-gg commented 6 years ago

@PTaylour yeah I have that checkbox ticked! Thank you for adding the tests

PTaylour commented 6 years ago

@reecelucas added you to review the tests I've just added here https://github.com/bbc/VideoContext/pull/103/commits/b4db86145af4c4cf2bfaf5b7b2e6210f3096ac0e

Couldn't get them to pass consistently locally. They were sometimes timing out on the goto page function. Have you seen this? Or likely an env thing?

PTaylour commented 5 years ago

regression tests now seem to be robust enough to merge this. Shall fire off a release now