Feature: WebCodecs + WebGL video

rhysmorgan134 / node-CarPlay

MIT License

131 stars 23 forks source link

Feature: WebCodecs + WebGL video #67

Closed gozmanyoni closed 9 months ago

gozmanyoni commented 9 months ago

This PR is based on https://github.com/codewithpassion/foxglove-studio-h264-extension/tree/main

I re-used the worker from there to directly decode h264 frames and draw them on a canvas using WebGL - this replaces jmuxer, and offloads decoding to hardware + a separate thread.

gozmanyoni commented 9 months ago

@rhysmorgan134 @steelbrain I also suspect with this we might not need the keyFrame interval

rhysmorgan134 commented 9 months ago

super super exciting! Hows the performance difference? I'll check it out as soon as I can

gozmanyoni commented 9 months ago

@rhysmorgan134 from what I have noticed:

video stream starts up faster (now need to mux at all, so i assume we get a stream as soon as there is a keyframe)
video overall is snappier
interface is more responsive (main UI thread is a lot less busy now)

Did not measure in any scientific way though - the worker has an internal FPS indicator so we could print that and also calculate fps with jmuxer.