1920x1080 stream with choppy video

jozefchutka commented 4 years ago

Hi Samir, thanks for the great lib!

I have recenlty played with jmuxer, but experienced very choppy video when used with ffmpeg desktop streaming and fullhd video. I put together a small demo to showcase:

demo.js (node demo.js)

const WebSocket = require('ws');
const spawn = require('child_process').spawn;
const shell = "ffmpeg -f gdigrab -framerate 30 -video_size 1920x1080 -i desktop "
    + "-an -c:v h264 -preset ultrafast "
    + "-f rawvideo -";

new WebSocket.Server({port:8080}).on('connection', function(ws) {
    spawn(shell, [], {shell:true}).stdout.on('data', data => {
        ws.send(data);
    });
});

demo.html

<!DOCTYPE html>
<html>
<head><script type="text/javascript" src="jmuxer.js"></script></head>
<body><video id="player" width="640" height="480" controls="false" autoplay muted></video></body>
<script>
window.onload = function startup() {
    const jmuxer = new JMuxer({node:'player', mode:'video', flushingTime:1});
    const ws = new WebSocket("ws://localhost:8080/");
    ws.binaryType = "arraybuffer";
    ws.addEventListener('message', event => {
        jmuxer.feed({video:new Uint8Array(event.data)});
    });
}
</script>
</html>

the rendered video looks as following: Clipboard01

When feeding the same stream into ffplay ffmpeg -f gdigrab -framerate 30 -video_size 1920x1080 -i desktop -an -c:v h264 -preset ultrafast -f rawvideo - | ffplay -i - it renders just ok.

Do you have any idea what can be done to make the redering complete?

jozefchutka commented 4 years ago

I managed to resolve it by restricting bitrate. It seems there is some kind of bottleneck on client side related to processing large amount of data on socket or on jmuxer side.

samirkumardas commented 4 years ago

Glad to hear. I will check it out when I get some time. I assume it is a WebSocket issue. I have also encountered data congestion issue with WebSocket and it is expected.

samirkumardas commented 4 years ago

Probable duplicate issue https://github.com/samirkumardas/jmuxer/issues/44

jozefchutka commented 4 years ago

Thanks for the update. I have checked with the latest jmuxer.min.js it seems improved but still not perfect. Now a larger part of image is rendered ok, however artifacts appears further down when streaming 1920x1080

samirkumardas commented 4 years ago

I just tested with 1920x1080 and found it good.

Sample file: sample_1920x1080.h264 You can test yourself https://samirkumardas.github.io/jmuxer/h264_player.html

Could you please share a clip for which you are having a problem so I can test myself?

jozefchutka commented 4 years ago

I am not able to replicate the issue using static file. I will need to investigate further. Keep the issue closed and consider resolved. Thanks

samirkumardas commented 4 years ago

Ok.

FYI, jMuxer considers each chunk/packet (i.e. data that you feed each time) as a complete buffer of the frame(s). If you feed buffer with partial frames, it may cause.

In future updates, I will try to handle it. For now, it should be handled before feeding e.g. on the Server-end

samirkumardas / jmuxer

1920x1080 stream with choppy video #35