search

mbebenita / Broadway

A JavaScript H.264 decoder.
Other
2.73k stars 424 forks source link

yuv to rgb performance issue #179

Open kimdimasan opened 6 years ago

kimdimasan commented 6 years ago

Hi I'm testing my live video streaming app that is using broadway. Using the chrome performance analyzer I've noticed that the bottle neck is the yuv to rgb transformation. I'm playing h264 720p stream and with or without webgl this stage takes 50 percent of cpu time at best. I was wondering is there an efficient h264 -> mjpeg transcoding algorithm. Cause h264 and jpeg have something in common in terms of using DCT and yuv space. So could we transcode h264 frames directly to simple jpeg images, that way we could simply show jpeg images using browser power by replacing image.src with jpeg binary. I know that there is an efficient mpeg1 -> mjpeg transcoder, but not sure about h264. Is it possible? I bet that would make broadway several times faster.

soliton4 commented 6 years ago

@kimdimasan that sounds awesome. make a pr and i can test it on my system to compare performance

kimdimasan commented 6 years ago

I don't know yet how to implement this transcoding and if it's possible, I know for sure that it could be done with phoboslab jmspeg (mpeg1->mjpeg). As for the profiling here what I've had: screenshot_20180701_224451

kimdimasan commented 6 years ago

It seems that the texture upload takes too much time in my case, may be it's a webgl issue

OllieJones commented 6 years ago

Have you tried disabling webGL and getting the decoder to generate RGBA and the player to draw it on canvases?

kimdimasan commented 6 years ago

Yes, I have tried it, it didnt help, the performance actually dropped significantly, I have ended up decoding all frames for full hd but render only every second or third one if it is too slow, dynamically deopping fps. Well its better to drop fps then experience a stuttering for me.

kimdimasan commented 6 years ago

My friend tried it on a better pc and it can handle 4k stream with low input fps (7 frames per second is what their are producing). On my pc it actually cant handle it not sure why. Using chrome.