tfabris / CrowCam

A set of Bash scripts to control and maintain a YouTube live cam from a Synology NAS.
GNU General Public License v3.0
4 stars 3 forks source link

Audio desynchronized with video. #14

Open tfabris opened 5 years ago

tfabris commented 5 years ago

Currently the audio stream is desynchronized from the video stream. The audio is several seconds earlier than the video.

To do:

tfabris commented 5 years ago

Reduced stream resolution from 1440p to 1296p.

Fixed the problem while still keeping a pretty nice sharpness increase above 1080p. Youtube only displays a 1080p stream, but it's sharper now since the source image is 1296p.

Why it delays the audio at 1440p I don't know, but I also don't care any more, so, closing issue.

tfabris commented 5 years ago

Still getting occasional situations where the audio is desynched from the video. Not every day, but some days. Example is 2019-04-16 9:40am - Crow scolds squirrel and the scolding is not synched up with the video.

To do: Try switching back down to 1080p and see if the problem goes away.

tfabris commented 5 years ago

Note: Checked the timestamp mentioned above (2019-04-16 9:40am) - Desynchronization issue was in the YouTube live stream DVR scrollback, but not in the Synology Surveillance Station hard copy. Jury is still out as to if it's in the archive on YouTube or not (not done processing yet).

tfabris commented 5 years ago

Desynchronization still exists when I set the camera feed to 1080p (from the camera controls), so 1080p doesn't actually solve the camera issue. With regard to the 2019-04-16 9:40am archive, the jury is still out (not done processing yet).

tfabris commented 5 years ago

A helpful tip in this thread says: https://telestreamforum.forumbee.com/t/h46z5j/youtube-stream-audio-out-of-sync:

"I've not noticed audio out of sync during the live stream, but when YouTube processes the video for playback, audio 4-5 seconds out of sync. Everything seemed fine until about 3-4 weeks ago.

To set the YouTube recorded video back into sync (has worked 9/10 times) I've been going to Edit>Audio>Revert to original .... takes a bit to process, but seems to re-sync the audio."

tfabris commented 5 years ago

Updates:

To do:

tfabris commented 5 years ago

There are no audio options to change, either on the camera or on the Synology camera software. The options are either "AAC Audio" or "Off". So we're stuck with that.

Trying

Note: Frame rate is default 25 on the camera itself but the Synology does not have a 25 option, it only has 30 or 22. I'm trying setting both of them to 22 so that they have a match, to see if that fixes the desynch issue.

tfabris commented 5 years ago

None of the above worked. Audio on YouTube is still desynchronized from the video.

To do:

Trying now, before putting the frame rate settings back:

tfabris commented 5 years ago

Rebooting the camera did in fact work:

Other notes:

To do:

tfabris commented 5 years ago

After the 10:22am reboot on 2019-04-19, there was a 3:17pm squirrel visit which was more or less synch'd. At most the audio was off by just a small fraction of a second, which is usually the best I can hope for with this system. So the camera reboot seems to fix the issue. So experiment with whether you can reboot the camera from its web interface after turning on the stream.

tfabris commented 5 years ago

I returned the resolution and frame rate settings back to their prior settings on the morning of Monday 2019-04-22. Currently they are at: 1296p/25fps camera. 1080p/30fps YouTube stream.

To do:

tfabris commented 5 years ago

Issue continues to recur intermittently. For instance I noticed it on 2019-04-27.

I went to investigate the reboot options and I noticed that the camera already has an auto-reboot feature built into itself. And I've already got it set to 3:30 am, the same time the LAN reboots.

Trying this now:

tfabris commented 5 years ago

To do:

tfabris commented 5 years ago

Since the last post about this (10:50 am 2019-04-28) I changed the camera auto reboot time. Also the camera has been rebooted to take care of other issues too. We've had good audio sync during all of this time. I've looked at video from 04/28 04/29 04/30 and 05/01 and the audio has been in sync.

My working theory is that, because I was bouncing the camera and bouncing the network at the exact same time of day, something was getting messed up. Now I am bouncing the camera an hour after the network bounces, and this seems to be working to keep video/audio in sync.

Closing this bug until I see a desynchronization again.

tfabris commented 5 years ago

On 2019-05-06 at 5:32 am there was a crow visit where the YouTube DVR scrollback had BETTER audio synchronization than the Synology Surveillance Station "Timeline" hard copy. Both were off a little bit, but YouTube was only off a fraction of a second, whereas the Synology Timeline was off at least a second or more.

Trying (wondering if this may also help issue #23):

tfabris commented 5 years ago

The issue of "Synology is desynchronized but YouTube is synched" continues to occur. But YouTube has been pretty good so far recently.

If YouTube is good, then I'm good, so I'm closing this bug for now and I'll reopen it if I see audio desynchronized on YouTube again.

tfabris commented 5 years ago

Reopening: Issue recurred on YouTube stream on May 10th, example here: https://youtu.be/auyhGvpjnU0?t=3985 This was early in the morning, so not terribly long after the camera auto-rebooted.

tfabris commented 5 years ago

This video starts out OK with the audio synchronized, but then the audio is out of synch by 2019-05-19 10:23 am. https://www.youtube.com/watch?v=ynFcxTtbNHY

The video was automatically bounced due to problems after 1pm or so. However that's hours later than this desynchronization issue. So the video bounce detection code is not helping to make sure the audio stays in synch, unfortunately.

tfabris commented 5 years ago

Most of the videos have their time synched just fine, and I can't see a solution for fixing this via a script. Perhaps I just need a better camera. Audio is desynchronized coming into Surveillance Station (surveillance station historical records are desynchronized always, YouTube is desyncrhonized only sometimes). So it's likely just the cheap ass camera. This is likely to be more related to issue #3 than anything else. Closing as "can't fix".

tfabris commented 5 years ago

In an attempt to fix issue #25 (camera rez changes from 1296p to 1080p sometimes) I tried setting things to the following:

Not sure if that worked in the long term yet. But I believe that this may have caused audio to be more frequently out of sync (issue #14 recurs).

To do: Experiment with setting it to:

tfabris commented 5 years ago

I tried setting everything to 1080p and I have the following conclusions:

I'm going to close bugs #25 and #14 now, leave everything at 1080p, and we'll reopen these bugs if needed later.

tfabris commented 5 years ago

Some things I have noticed:

tfabris commented 4 years ago

Reopening. The new Dahua camera still has an audio desynch issue. Or, more specifically, the audio deaynch issue is not due to the camera, it's got to be a bug in Surveillance Station and/or YouTube.

Ideas:

tfabris commented 4 years ago

Tried:

Immediately after changing this setting, it caused the stream to go temporarily offline and made CrowCam.sh bounce the stream. After the stream came back from the bounce, the audio on the YouTube channel was perfectly synchronized.

This might not mean much. I the past, I seem to recall that stream bounces would resynch the audio as well. Live with this a while and see if it stays good.

tfabris commented 4 years ago

Indeed, after changing the setting, even though things had started out fine, video from later that same day was desynchronized. (Note: Checked the video on YouTube the following day, after YouTube was done fully processing the video. Also checked the Surveillance Station recording of the same section, and it was correctly synchronized.)

I'm not the only person experiencing this problem. Here is a message on the Synology community forum with the exact same problem: https://community.synology.com/enu/forum/3/post/125996

tfabris commented 4 years ago

Submitted Synology support ticket number 2464131 .

tfabris commented 4 years ago

Synology sent a list of suggestions with many things to try. In addition to the things they listed, also try:

tfabris commented 4 years ago

On 2020-02-27, starting at approx 11am-noon ish, I began trying to implement some of the suggestions:

Unable to do:

Still to do:

tfabris commented 4 years ago

Audio desynchronization repeated on this video: https://youtu.be/tmfEU3HEMY8?t=3078 clearly audible at 2020-02-29 07:31:26 am. Though, since that was occurring this morning and mayyyyyybe youtube wasn't done fully processing the video file, it's worth checking back to see if the problem is fixed later (though note that the video might be deleted after 10 days).

Set the video latency to "Low Latency" (originally it was Normal Latency) on 2020-02-29 at approx 5:54 pm.

tfabris commented 4 years ago

Setting latency to "Low Latency" didn't seem to fix the issue:

Timing correct: https://youtu.be/eq_cx-c7g-Y?t=1531 Timing incorrect (later time point in the same video): https://youtu.be/eq_cx-c7g-Y?t=13417

To do: Try to adjust to the lowest latency setting (there are three of them, try the third setting).

tfabris commented 4 years ago

Switched to Ultra Low Latency on 2020-03-03 at approx 1:04 am. Let's see how that does.

tfabris commented 4 years ago

Timing is off on this video: 2020-03-04 2:50:43pm - https://youtu.be/4FSGaDLZEs4?t=8435 So the Ultra Low Latency did not fix the problem.

New cable modem (comcast-leased Cisco DPC3941T) and new DHCP router (Linksys WRT wrt3200acm) installed on 2020-03-05. Downstream bandwidth has improved, but not upstream (still 6mbps). Let's see how this does.

tfabris commented 4 years ago

Investigate video timing after YouTube is done processing this video: 2020-03-06 3:48:18pm https://youtu.be/LbBQBpTlFZU?t=12068 When watching this one "live", the video was synchronized when early in the video, and desynchronized when late in the video, at the indicated time position. Recheck this video in a day or two, so that we can be sure the sound is still desynchronized after the video has been fully post-processed by YouTube.

If true, then upgrading router and cable modem hardware did not fix the problem.

Update: True. Video is desynchronized, even after YouTube is done post-processing.

tfabris commented 4 years ago

To do:

tfabris commented 4 years ago

I haven't seen a recurrence of the problem in the last three days (Edit: it took about 5 days for it to recur). Here are the things I have done in that time. Keep in mind that the problem was always intermittent for me, so these things might have nothing to do with it; perhaps the problem will still recur soon... we shall see. For example, maybe Comcast has been improving their service and fixing problems recently, due to increased customer usage, due to Covid-19, and maybe that is the reason it's been good for the last few days.

Update: None of these things fixed the problem, see below.

tfabris commented 4 years ago

To do:

tfabris commented 4 years ago
tfabris commented 4 years ago

Trying suggestion from Synology:

To do:

tfabris commented 4 years ago

The audio desynchronization issue recurred on 2020-04-06, several hours after I implemented the suggested NTP change. The issue was clearly noticeable at 4:56:23 pm (squirrel lands on feeder and you hear the sound of him landing on the feeder before he does it on video).

On this stream, I have traced it down to the exact second that the problem occurs. It occurs at 04:55:25 pm. You can see it happen if you watch the "seconds" of the time counter on the screen. There is a glitch in the video where the video freezes for a couple of seconds at that time, while the sound continues. After the glitch, the video and audio are desynchronized, but you might not be able to notice the issue until 4:56:23 pm when the squirrel jumps back onto the roof of the feeder.

Links to time points in the video stream:

I have looked at my router's DD-WRT YaMon hourly data usage on the network during that time period, and there is no unusual data usage during that period of time. No data spikes in the hourly report for the 4pm hour. There is a large file download which occurs during the 5pm hour on that afternoon, but the desynchronization issue occurred before that download had started.

I have checked the Synology NAS logs during that time period and there are no log entries for that time, nothing is expected to have gone wrong during that period.

The other user on the Synology forum who is experiencing the same problem has reported that his camera NTP settings were already set to point to the Synology, and it did not fix the issue for him: https://bit.ly/39OenPP

The other user on that Synology forum says that he has already ruled out all other networking problems, because he has no audio issues if he streams the video directly from the camera to YouTube using built-in software on the camera. He only has the problem when the Synology software is used to stream the video to YouTube.

So this must be a bug in Synology surveillance station software which streams the video. Synology must be doing something to the rtmp stream, which intermittently causes problems with the video/audio data stream.

tfabris commented 4 years ago

Interesting: Changing the camera NTP settings, though they didn't fix the audio problem, has revealed a new bug in the Synology NTP server software.

After I set the Surveillance station configuration to use the Synology as its NTP server, the camera's timestamp (the time that is baked into the video feed) occasionally displays an incorrect time. The minutes are correct but the hours are 7 hours off. I can only assume that the timezone, as supplied by the Synology NTP server, is flipping back and forth. The camera will occasionally go to the wrong time stamp and then later go back to the correct time stamp.

Here is an example. At 8:15:42 am, the camera flips to display an incorrect time of "1:15:42" am: https://bit.ly/2VfDKVk At 8:35:42 am, the camera flips back to displaying the correct time: https://bit.ly/2Rk6o6U

This issue never occurred before I had changed the setting, so that proves this is a Synology bug, not a camera bug.

I am now (2020-04-08 11:21am PDT) resetting the camera back to its original NTP configuration (pool.ntp.org) so that this issue no longer occurs.

tfabris commented 4 years ago

Synology support worked on it remotely and installed a patch that was supposed to resynchronize the audio at intervals (not sure what, exactly, I didn't get details). They did this apparently on 2020-03-13 at approximately 3am ish (Seems that it was across the 3:30am reboot boundary, they had to re-log-in, sorry guys), so investigate if the problem resolved itself after this date/time. They do not expect that it has resolved itself, they say that they think the problem remains after the patch.

To check this, I need YouTube to be working OK. At the moment, I'm getting a lot of spotty performance off of YouTube in terms of downloading and viewing video. I think that uploaded stream video is still working as expected but I'm not certain.

Initial investigation seems to indicate that, indeed, the problem still remains. This is the video from that morning: https://www.youtube.com/watch?v=LYPck6YYtHw and initial looks indicate the problem is still there. Note: I double checked it later, after YouTube has had a a few hours to fully process the video file, sometimes there's still processing in-progress if I view the archived video too soon after the stream has ended. It still had the problem.

tfabris commented 4 years ago

Synology says they thought the issue should be resolved, but the issue recurs at these time points today:

2020-04-20 9:46 am Squirrel: https://youtu.be/IRL6q-792rc?t=13870 At 10:01 am squirrel still seems synched

Desynch happens some time during this hour but there are no animals on screen to check sync.

2020-04-20 11:03:01 am Steller's Jay (audio desynchronized) : https://youtu.be/IRL6q-792rc?t=18465 2020-04-20 11:18:05 am Crow (audio desynchronized): https://youtu.be/IRL6q-792rc?t=19374

2020-04-20 11:34:51 am on the screen: https://www.youtube.com/watch?v=IRL6q-792rc&feature=youtu.be&t=20377 2020-04-20 11:54:00 am on the screen: https://www.youtube.com/watch?v=IRL6q-792rc&feature=youtu.be&t=21531 2020-04-20 12:32:44 pm on the screen: https://www.youtube.com/watch?v=IRL6q-792rc&feature=youtu.be&t=23852 2020-04-20 12:33:11 pm on the screen: https://www.youtube.com/watch?v=IRL6q-792rc&feature=youtu.be&t=23882

tfabris commented 4 years ago

Issue continues to occur. Here is issue occurring all morning, while a Synology support tech was logged in:

Issue occurring already, at the start of the stream, first thing in the morning, shortly after the stream went live: 2020-04-21 6:01:39 am - https://youtu.be/SoCew4k-2No?t=388

Then: 2020-04-21 7:23:09 am - Login from Synology support is registered in the Synology log.

Issue is still audible a few minutes, and hours, later: 2020-04-21 7:28:26 am - https://youtu.be/SoCew4k-2No?t=5594 2020-04-21 7:42:12 am - https://youtu.be/SoCew4k-2No?t=6425 2020-04-21 7:50:14 am - https://youtu.be/SoCew4k-2No?t=6902 2020-04-21 9:01:17 am - https://youtu.be/SoCew4k-2No?t=11158 2020-04-21 9:50:35 am - https://youtu.be/SoCew4k-2No?t=14127 2020-04-21 10:50:34 am - https://youtu.be/SoCew4k-2No?t=17726 2020-04-21 10:51:34 am - https://youtu.be/SoCew4k-2No?t=17786 2020-04-21 10:57:33 am - https://youtu.be/SoCew4k-2No?t=18136 2020-04-21 10:57:54 am - https://youtu.be/SoCew4k-2No?t=18166

There were no crowcam script errors in the Synology log except for a brief "500" error in the YouTube API at 7:52 am, though I see no problems or hiccups in the video feed during that time (apart from the desynchronization).

The entire stream stayed this way (desynchronized) until the midday stream bounce occurred, and then it was fixed after that bounce: https://youtu.be/beC_4jAxlOg?t=1735

tfabris commented 4 years ago

More updates: Synology says more work was done, an additional patch was applied, and to check it. They said that the "camera responded to Surveillance Station in ways that did not line up with the camera profile that it was configured to use. Based on their testing, they tried changing the camera profile from how it had been configured to the model of camera it appeared to be behaving as - in this case, from IPC-HDBW5831R-ZE to IPC-HDW5830R-Z" and wanted to be sure that I had the model number right. The camera was bought from https://www.amazon.com/gp/product/B077QTWQ7G/ and it says it's a white label OEM model so I can't be 100 percent sure it's not just the lesser camera. The camera's internal web page does not show the camera model number on the screen, so I don't know. However, I'm certain it's not a IPC-HDW5830R-Z because I looked up a picture of one of those on the web and it looks physically different.

Syno Logins registered at: 2020-04-22 19:22:36 2020-04-23 03:28:42 2020-04-23 03:29:30 2020-04-23 03:57:56 2020-04-23 03:58:17 2020-04-23 04:53:18 2020-04-23 04:54:19

Syno login registered at: 2020-04-23 21:09:03

Problem recurred in this stream: https://www.youtube.com/watch?v=S4FS1AQIIic 2020-04-24 05:54:37 am - Two crows, audio is working correctly: https://bit.ly/2VCDi5c 2020-04-24 06:06:06 am - Timestamp on video skips a little bit, possible start of audio desynchronization problem: https://bit.ly/2KF3lCc 2020-04-24 06:06:06 am - Crowcam Cleanup logs that it performed a deletion of an old video from YouTube. 2020-04-24 06:28:57 am - Squirrel - audio may be desynchronized, but hard to tell: https://bit.ly/2KGYZKR 2020-04-24 06:40:21 am - Squirfuffle - audio definitely desynchronized: https://bit.ly/3bFwqcX 2020-04-24 07:58:19 am - Two crows - audio definitely desynchronized: https://bit.ly/3aEutvO

tfabris commented 4 years ago

Current status:

Issue still occurring on Sunday morning , example: 2020-04-26 07:17:10 am - https://youtu.be/Q8JeFcXoC2I?t=5523

Logins from syno techs on that Sunday night at: 2020-04-26 19:22:11 2020-04-27 01:26:10 2020-04-27 02:51:06

Noticed that there had been changes to configuration of camera on Monday 04-27 during the day:

Audio is working correctly at this time point: 2020-04-27 6:12:04 am: https://youtu.be/fwnG3b9ocBg?t=1615

Audio is desynchronized at these time points: 2020-04-27 6:22:00 am: https://youtu.be/fwnG3b9ocBg?t=2212 2020-04-27 6:29:54 am: https://youtu.be/fwnG3b9ocBg?t=2684 2020-04-27 07:23:08 am: https://youtu.be/fwnG3b9ocBg?t=5879 2020-04-27 07:27:25 am:https://youtu.be/fwnG3b9ocBg?t=6135

Audio has a dropout at this time point, audio is silent for a moment, but audio is still desynchronized after the dropout: 2020-04-27 07:45:08 am: https://youtu.be/fwnG3b9ocBg?t=7201

tfabris commented 4 years ago

Status:

2020-04-30 00:19:14 am - Synology tech login. 2020-04-30 00:22:01 am - Synology reportedly "applied a new patch to Surveillance Station that should lower the frequency of the video stream restarting, and ideally should reduce the impact this has on the video stream"

Issue recurred the following morning:

2020-04-30 05:44:40 am - Crows, sound is synchronized correctly: https://youtu.be/YB7LeVDlvrM?t=268 2020-04-30 05:46:22 am - Crow, sound is synchronized correctly: https://youtu.be/YB7LeVDlvrM?t=370

Sound becomes desynchronized somewhere between 5:46 am and 6:07 am.

2020-04-30 06:07:37 am - Squirrel, sound is desynchronized (though hard to tell unless you know exactly what to listen for): https://youtu.be/YB7LeVDlvrM?t=1647 2020-04-30 06:32:28 am - Squirrel, sound is desynchronized: https://youtu.be/YB7LeVDlvrM?t=3139 2020-04-30 06:47:24 am - Squirrel, sound is desynchronized: https://youtu.be/YB7LeVDlvrM?t=4033

tfabris commented 4 years ago

Most recent logins by Syno techs:

Audio desynchronizations noticed:

tfabris commented 4 years ago

Desynchronization issue still occurs. Most recent logins by Syno techs is still this: 2020-05-05 00:27:34 AM 2020-05-05 05:23:21 AM

Example of recent desynchronization: 2020-05-17 06:24:23 pm: https://bit.ly/3gaTrH5

tfabris commented 4 years ago

Rechecked the support thread, and actually on 2020-05-05 I had missed a message. They have supplied me with information on a version of the FFMpeg binary that I can use on the Synology to stream directly to YouTube via FFMpeg instead of through the "Live Broadcast" feature of Surveillance Station. This is complicated but theoretically do-able. I've made local notes on how to do it, so I can start working on the code to try it.

tfabris commented 4 years ago

Current status: