rdp / screen-capture-recorder-to-video-windows-free

a free open source windows "screen capture" device and recorder (also allows VLC/ffmpeg and others to capture/stream desktop/audio)
https://github.com/rdp/screen-capture-recorder-to-video-windows-free/releases
Other
2.11k stars 463 forks source link

Permanent buffering delay when unchanging screen while muxing with audio using ffmpeg #66

Closed DuBistKomisch closed 9 years ago

DuBistKomisch commented 9 years ago

Hi, I've run into a weird issue.

Our situation: The client runs ffmpeg.exe and takes inputs from screen-capture-recorder, encodes with libx264, and outputs as mpegts (or matroska or flv, all show this issue) to a TCP socket over LAN. The server is a gstreamer pipeline on another machine listening on the TCP socket and decoding and playing the video.

The issue: Suppose the client machine is playing a fullscreen video, then pauses it. Or just goes to the desktop and leaves the mouse still. Anything which leaves the screen static. What happens while it's paused, is a delay gradually gets added to the stream. It seems to max out at exactly 10 seconds.

My analysis: I've configured gstreamer to just play frames as fast as it can as it receives them. If after the delay has started, I go to the command prompt ffmpeg is running in and press ctrl+c, ffmpeg sends out all of the delayed data, and the final 10 seconds plays extremely fast on gstreamer. Therefore, this seems like an issue internal to buffering in ffmpeg or screen-capture-recorder. I also wrote a quick script using netcat to print out a . for every 100 bytes received, and it completely freezes a few seconds after the screen stops changing, then resumes, not sure what's going on with that.

Here's the ffmpeg command line I'm using: .\ffmpeg.exe -framerate 10 -f dshow -i video=screen-capture-recorder:audio=virtual-audio-capturer -vf scale=1280:720 -r 10 -vcodec libx264 -pix_fmt yuv420p -tune zerolatency -preset ultrafast -crf 30 -async 1 -strict -2 -acodec aac -f mpegts tcp://192.168.2.230:12802 If the audio input is removed, or the screen is always changing, it works great with < 1 second delay.

Perhaps screen-capture-recorder doesn't output anything if the screen isn't changing? Otherwise something else weird affecting the muxers? I don't know enough about ffmpeg to really analyse this.

rdp commented 9 years ago

what if you use a different video codec?

On Fri, Sep 18, 2015 at 3:12 AM, Jake Barnes notifications@github.com wrote:

Hi, I've run into a weird issue.

Our situation: The client runs ffmpeg.exe and takes inputs from screen-capture-recorder, encodes with libx264, and outputs as mpegts (or matroska or flv, all show this issue) to a TCP socket over LAN. The server is a gstreamer pipeline on another machine listening on the TCP socket and decoding and playing the video.

The issue: Suppose the client machine is playing a fullscreen video, then pauses it. Or just goes to the desktop and leaves the mouse still. Anything which leaves the screen static. What happens while it's paused, is a delay gradually gets added to the stream. It seems to max out at exactly 10 seconds.

My analysis: I've configured gstreamer to just play frames as fast as it can as it receives them. If after the delay has started, I go to the command prompt ffmpeg is running in and press ctrl+c, ffmpeg sends out all of the delayed data, and the final 10 seconds plays extremely fast on gstreamer. Therefore, this seems like an issue internal to buffering in ffmpeg or screen-capture-recorder. I also wrote a quick script using netcat to print out a . for every 100 bytes received, and it completely freezes a few seconds after the screen stops changing, then resumes, not sure what's going on with that.

Here's the ffmpeg command line I'm using: .\ffmpeg.exe -framerate 10 -f dshow -i video=screen-capture-recorder:audio=virtual-audio-capturer -vf scale=1280:720 -r 10 -vcodec libx264 -pix_fmt yuv420p -tune zerolatency -preset ultrafast -crf 30 -async 1 -strict -2 -acodec aac -f mpegts tcp:// 192.168.2.230:12802 If the audio input is removed, or the screen is always changing, it works great with < 1 second delay.

Perhaps screen-capture-recorder doesn't output anything if the screen isn't changing? Otherwise something else weird affecting the muxers? I don't know enough about ffmpeg to really analyse this.

— Reply to this email directly or view it on GitHub https://github.com/rdp/screen-capture-recorder-to-video-windows-free/issues/66 .

DuBistKomisch commented 9 years ago

I tried using libvo_aacenc instead of aac and the issue disappeared, strange. I guess aac is experimental for a reason. Thanks anyway, closing.

DuBistKomisch commented 9 years ago

Actually it still happens... I'll try another video encoder.

rdp commented 9 years ago

I wonder if somehow the muxer is "waiting" for video packets? What if you run it with -loglevel verbose? example output please?

On Fri, Sep 18, 2015 at 3:12 AM, Jake Barnes notifications@github.com wrote:

Hi, I've run into a weird issue.

Our situation: The client runs ffmpeg.exe and takes inputs from screen-capture-recorder, encodes with libx264, and outputs as mpegts (or matroska or flv, all show this issue) to a TCP socket over LAN. The server is a gstreamer pipeline on another machine listening on the TCP socket and decoding and playing the video.

The issue: Suppose the client machine is playing a fullscreen video, then pauses it. Or just goes to the desktop and leaves the mouse still. Anything which leaves the screen static. What happens while it's paused, is a delay gradually gets added to the stream. It seems to max out at exactly 10 seconds.

My analysis: I've configured gstreamer to just play frames as fast as it can as it receives them. If after the delay has started, I go to the command prompt ffmpeg is running in and press ctrl+c, ffmpeg sends out all of the delayed data, and the final 10 seconds plays extremely fast on gstreamer. Therefore, this seems like an issue internal to buffering in ffmpeg or screen-capture-recorder. I also wrote a quick script using netcat to print out a . for every 100 bytes received, and it completely freezes a few seconds after the screen stops changing, then resumes, not sure what's going on with that.

Here's the ffmpeg command line I'm using: .\ffmpeg.exe -framerate 10 -f dshow -i video=screen-capture-recorder:audio=virtual-audio-capturer -vf scale=1280:720 -r 10 -vcodec libx264 -pix_fmt yuv420p -tune zerolatency -preset ultrafast -crf 30 -async 1 -strict -2 -acodec aac -f mpegts tcp:// 192.168.2.230:12802 If the audio input is removed, or the screen is always changing, it works great with < 1 second delay.

Perhaps screen-capture-recorder doesn't output anything if the screen isn't changing? Otherwise something else weird affecting the muxers? I don't know enough about ffmpeg to really analyse this.

— Reply to this email directly or view it on GitHub https://github.com/rdp/screen-capture-recorder-to-video-windows-free/issues/66 .

DuBistKomisch commented 9 years ago

Here's a log at -loglevel verbose: https://paste.ee/p/7MTbZ

The interesting part is at line 15038, where audio packets stop appearing until line 15332. Seems like the muxer waiting for audio packets could be the problem actually, so perhaps it's an issue with virtual-audio-capturer instead?

I'm testing this by starting the stream in powershell, alt-tabbing to youtube open in a chrome tab, hitting play, watching for a bit, then pausing for ~10 seconds, and then hitting play again.

rdp commented 9 years ago

OK so you are able to repro it? what os are you on?

On Thu, Sep 24, 2015 at 12:00 AM, Jake Barnes notifications@github.com wrote:

Here's a log at -loglevel verbose: https://paste.ee/p/7MTbZ

The interesting part is at line 15038, where audio packets stop appearing until line 15332. Seems like the muxer waiting for audio packets could be the problem actually, so perhaps it's an issue with virtual-audio-capturer instead?

I'm testing this by starting the stream in powershell, alt-tabbing to youtube open in a chrome tab, hitting play, watching for a bit, then pausing for ~10 seconds, and then hitting play again.

— Reply to this email directly or view it on GitHub https://github.com/rdp/screen-capture-recorder-to-video-windows-free/issues/66#issuecomment-142819314 .

DuBistKomisch commented 9 years ago

Yes, it happens consistently. This is on a Windows 8/8.1 64-bit laptop.

rdp commented 9 years ago

did you ever figure this out?

On Fri, Sep 25, 2015 at 12:34 AM, Jake Barnes notifications@github.com wrote:

Yes, it happens consistently. This is on a Windows 8/8.1 64-bit laptop.

— Reply to this email directly or view it on GitHub https://github.com/rdp/screen-capture-recorder-to-video-windows-free/issues/66#issuecomment-143141655 .

DuBistKomisch commented 9 years ago

No, we're happy with only video (no audio) as a temporary workaround. I may just try using separate unmuxed video and audio streams if I can't figure out what's going on with the audio capturer.

DuBistKomisch commented 9 years ago

So it turns out that this is just a symptom of the underlying silence capturing problem with virtual-audio-capturer (e.g. https://github.com/rdp/virtual-audio-capture-grabber-device/issues/10 and https://github.com/rdp/virtual-audio-capture-grabber-device/issues/13). Specifically our issue disappears when silence.exe from the blog post you linked is running. I'll integrate something into our application to do the same job, which should be a good enough workaround.

EzeKees commented 2 years ago

Sorry to revive this, but this issue is still happening to this day.