search

phoboslab / jsmpeg

MPEG1 Video Decoder in JavaScript
MIT License
6.3k stars 1.43k forks source link

Is it feasible to perform YUV format conversion on the server side? #426

Closed syjjx closed 3 months ago

syjjx commented 3 months ago

One of the most computationally intensive tasks for an MPEG1 decoder is the color conversion from MPEG's internal YUV format (Y'Cr'Cb to be precise) into RGBA so that the browser can display it.

So if the network transmission between my server-side and client-side (browser) is reliable and fast, can I convert YUV format to RGBA and let the server handle it, so that the browser doesn't need to perform the conversion logic and complex rendering?

phoboslab commented 3 months ago

Well, you can uncompress MPEG1 on the server and send raw RGB data over websocket... but then you don't need this library.

For what it's worth, the YUV->RGB conversion is essentially free when the WebGL renderer is enabled (default) and WebGL is supported by the browser (which it is, if you're not on IE6 :))

What's your use case? If you have a compute constrained client, I'd probably first lower the video resolution, then quality and then framerate, until the client can keep up.

syjjx commented 3 months ago

Thank you for your reply, and thank you for developing this library.

I usually use Tesla because my MCU has an Atom CPU, which cannot play high-quality videos. Currently, the quality of the videos I stream is 720P, 25FPS, with a maximum bitrate of 2.5M. Despite this, I still encounter dropped frames. Of course, I have also tested using Tesla with an AMD CPU, and it can smoothly decode 1080P, 30FPS, 8Mbit videos. On the server side, I run it on a mobile router, which provides WiFi network access for the vehicle.

Based on this requirement, I came up with the idea of letting the server side decompress MPEG1, which can help alleviate the computational pressure on the vehicle.

https://github.com/phoboslab/jsmpeg/assets/39109231/ca81858b-fc3f-4479-abd5-b51e559e9c3e