search

Arcus92 / libpgs-js

Renderer for graphical subtitles (PGS) in the browser.
MIT License
5 stars 1 forks source link

Re-write in C using WASM #6

Closed Arcus92 closed 1 month ago

Arcus92 commented 3 months ago

Parsing the PGS files could be slow on low-end devices using JavaScript. I'm working on a C implementation using WebAssembly with Emscripten and WebWorker support.

ferferga commented 3 months ago

I'd advise being cautious about this one. Currently, WASM works flawlessly in Firefox (regarding raw performance), but in Chromium browsers it's really slow. You can do a quick test here with an intensive workload, it's night and day: https://ffmpegwasm.netlify.app/docs/getting-started/usage/

In my thesis I did quite a few tests with WASM and I came to the conclusion that WASM doesn't give noticeable improvements for performance-critical scenerios, in it's current state it's main purpose is for those applications that can't be ported/developed in JavaScript, like our libass in SubtitlesOctopus or Google Earth and Photoshop (for some public app examples).

If you want to do some benchmarks in a real application, you can install blurhash-wasm into our current web client or Jellyfin Vue and compare it with the current WebWorker implementation (there might be differences in the algorithm implementation, but I think it's one simple enough test that you could try)

What if you split up the current worker into 2 workers? One for OffscreenCanvas and another for the other? So they don't block each other. If it's still slow in some devices, you can still give it a go with memoization.

Arcus92 commented 3 months ago

@ferferga Thanks for your advice. You are just confirming my doubts on this topic. I spend the last week looking into and learning WebAssembly and discovered the big overhead WASM is adding to this project. The only area I saw an issue was allocating many Color instances when parsing the palette. Structs could solve this. However, most subtitles only use a hand full of colors. Also browsers are optimized to handle many instances in an array of the same type. For fun I'll try to store the pixel data in one big byte array and measure if it makes any difference. I also got a Raspberry PI 4 to test. If I can view a 1080p video with the current PGS implementation on Chromium and Firefox without adding to much CPU load I'll call it a day for now.

Arcus92 commented 2 months ago

Made some good changed in #9 to improve the performance. It runs smoothly with 1080p - 10 Mbps on a Raspberry PI 4 (4gb, no overclocking) on Chromium. Firefox already struggled with 1080p video streaming. Subtitle rendering or clearing can make the video lag. Pixel data encoding is not the issue though. clearRect and drawImage are blocking the browser - even from a web-worker. I guess the software renderer is just that slow.