Weird bug I've been noticing (recording hangs)

Hi. Something weird I've been noticing while checking my daily C4 and CB mp4s is that some recordings (very few, but it's happening) from some streamers get stuck, they are like incomplete. How I noticed that? Because the streamer have been online for 2h and I only have 15 min of recording. I don't know if it's due to a bug, but to give an example of what it's happening: 1) the streamer goes online 2) the downloader starts recording as intended 3) after a while, for some reason the recording hangs. But the downloader does not detect any streamer status change, she is still "online" but not recording anything 4) the streamer is still broadcasting but the algorithm is not recording because it got stuck in the previous step, so we lose that model broadcast (for example if the model streams for 2h but the recording got stuck at min 15, we lost 1h 45min of streaming) 6) The streamer goes offline but the downloader keep showing online for the whole day 7) because the bot doesn't detect any offline status, there's no "stop" signal, so the ffmpeg.exe process is open for the infinity and the file can't be played because ffmpeg.exe is using it. 8) because the ffmpeg.exe process is stuck endlessly, you have to kill it manually with task manager (it doesn't terminate with the stop command either) so the file we get is a corrupted mp4 of an incomplete recording, because the rest of the stream could not be recorded as it got stuck at minute 15 (for example)

I don't know why is this happening. I thought that maybe the streamer disconnects for a sec because of an internet issue or something and the downloader doesn't detect it and gets stuck ¿? I really have no idea. It's just happening with 2% of the recordings but it's there and I already lost a couple of important streams. I don't have console output of the issue because I have no idea when the problem is reproduced, probably hours before I made the daily review of the recordings. What do you think? Cheers.

This have mostly two reasons:

The one you already mentioned. If a stream shortly wents offline it is possible that there is something like a "silent" stream. So the stream still exists at the streaming server but it has no output. That's something from time to time also happens with Youtube Streams. ffmpeg captures "something that's nothing" and at some point stops it self (I had situation where these took hours). Mostly this situation is indicated by messages like (frame received already or so). I think you corrupt your videos yourself by killing ffmpeg - keep in mind that it's not simple on Windows. This can't be fixed that simple. There would be the need to analyze ffmpeg to stop it depending on flexibel conditions. As long as a spawned ffmpeg runs for a streamer the online status isn't checked. So if ffmpeg hangs the streamer thread hangs.
Slow internet connection leads to similiar effects. in this case ffmpeg from time to time stuck. The whole task StreaMonitor does is a material war (disk and network). I would recommand just running it on a 1GBit system with in the best cast static public IP and good disk I/O. Everything else will always lead to big problems. Never had a good performing box at dial-up lines, behind NATs and so on.

Best Regards

I forgot to say that I discovered that there were recordings hanging for various reasons:

1) When I do my daily file cleaning, when I play some videos I got the typical error when you try to play a recording in progress (unfinished), but when I checked streamer's website I saw that she had been offline for hours (a whole day sometimes). So to delete or play that file (some of them get hung after more than 1gb of recording, some others just after few mins) you have to stop the recording, but as it is stuck the stop command does not work in this specific case, so there is no choice but to kill the process manually, getting the consecuent corrupt video. But the problem I want to point out is not the corrupted files (which are consecuence of killing ffmpeg as we discussed last month) but that 1% of the recordings getting stuck only with streamonitor.
2) why I deduce this "only" happens with streamonitor? Because when I check a certain webcam recording download site, I could see that that streamer recording was there, completed (for example 2h session), while my streamonitor only recorded 15 min before getting stuck. This means there was output and the problem it's probably on the recording software side rather than in the streamer's, otherwise that recording wouldn't exist (no output) in that website either.

Hope this helps. Cheers.

PS: I have very fast cable internet, so I don't think the issue is there.

I forgot to say that I discovered that there were recordings hanging for various reasons:

1. When I do my daily file cleaning, when I play some videos I got the typical error when you try to play a recording in progress, but when I checked streamer's website I saw that she had been offline for hours (a whole day sometimes). So to delete or play that file (some of them get hung after more than 1gb of recording, some others just after few mins) you have to stop the recording, but as it is stuck the stop command does not work in this specific case, so there is no choice but to kill the process manually, getting the consecuent corrupt video. But the problem I want to point out is not the corrupted files (which are consecuence of killing ffmpeg as we discussed last month) but that 1% of the recordings getting stuck only with streamonitor.

2. when accessing a certain webcam recording download site, I could see that that streamer recording was complete (for example 2h session), while my streamonitor only recorded 15 min before getting stuck. This means there was output and the problem it's probably on the recording software side rather than in the streamer's, otherwise that recording wouldn't exist (no output) in that website either.

Cheers.

This sounds more like the thing explained in point 2.

StreaMonitor has no control over the recording process. It's just a subprocess manager for ffmpeg. So the recording problems happen to ffmpeg. Because there is actually no monitoring what ffmpeg is doing we wouldn't know when it stucks. And also don't know whether it only stucks or is hang in sense of "crashed process".

Because the rate is not that high I would assume network related stuff. So inconsistent dial-up lines, NAT boxes that kill connections in a strange way which leads to hanging ffmpeg processes. If cable means Internet via the TV cable that's the worst environment for stuff like this. For Europe aspecially they are the lines which are heavily NATed, have wild IPv4/6 conversion stuff in the middle and for technical reasons can have problems in peak situations because the wire must be shared in a few circumstances. I always had the biggest issues at TV cable lines with long time communication processes. Also stuff in the local network like W-LAN, D-LAN (Powerline) and so on is all crap for such boxes. I guarantee this leads to hanging ffmpegs. The best situation is something like glas fibre with static public IPv4 access. Everything else directly wired with at leas 1GBit Ethernet Cat 6 and up. In the middle only well working NAt gateways which guarantee consistent backflow and doesn't kill or corrupt connections. HLS is already very frugal but today we also have stuff like keeping HTTP connections open. I don't know how ffmpeg opperates in detail at HTTP level but it's really hardly sensible to the network behaviour aspecially when it opperates as a client.

Also it would be possible that this is related to the host system. But this would be a search of a needle in the haystack. In general the stuff runs best in Unix.

lossless1024 / StreaMonitor

Weird bug I've been noticing (recording hangs) #42