search

roflcoopter / viseron

Self-hosted, local only NVR and AI Computer Vision software. With features such as object detection, motion detection, face recognition and more, it gives you the power to keep an eye on your home, office or any other place you want to monitor.
MIT License
1.77k stars 179 forks source link

invalid audio pts, packets dropped #656

Open ceewanna opened 1 year ago

ceewanna commented 1 year ago

I have been observing the playback of vdo files for quite a while. There seems to be quite a number of these invalid audio pts. Initially I thought it was caused by wifi problem or gpu problem. However, after pinging and adding gpu card, the issue remains. These vdo files will often get hiccups or stumbled from time to time. I took a sample from mpv playback of a file in this case. There seems to be quite a number received packets dropped while recording the video file.

Playing: rec/ch30_barn/2023-08-06/174024.mp4 (+) Video --vid=1 () (hevc 2560x1440 19.569fps) (+) Audio --aid=1 () (aac 1ch 16000Hz) AO: [pulse] 16000Hz mono 1ch float VO: [gpu] 2560x1440 yuv420p AV: 00:00:00 / 00:00:46 (1%) A-V: 0.000 Invalid audio PTS: 0.391937 -> 0.589500 AV: 00:00:00 / 00:00:46 (2%) A-V: -0.118 Dropped: 1 Invalid audio PTS: 1.190500 -> 1.832250 AV: 00:00:02 / 00:00:46 (6%) A-V: -0.107 ct: -0.075 Dropped: 3 Invalid audio PTS: 2.984688 -> 3.218313 AV: 00:00:03 / 00:00:46 (7%) A-V: -0.113 ct: -0.068 Dropped: 3 Invalid audio PTS: 3.466625 -> 3.574625 AV: 00:00:03 / 00:00:46 (8%) A-V: -0.093 ct: -0.084 Dropped: 3 Invalid audio PTS: 3.790812 -> 3.893375 AV: 00:00:05 / 00:00:46 (11%) A-V: -0.081 ct: -0.130 Dropped: 3 Invalid audio PTS: 5.357875 -> 5.549500 AV: 00:00:06 / 00:00:46 (13%) A-V: -0.014 ct: -0.147 Dropped: 3 Invalid audio PTS: 6.430438 -> 7.059000 AV: 00:00:06 / 00:00:46 (13%) A-V: -0.014 ct: -0.149 Dropped: 3

roflcoopter commented 1 year ago

I am no expert on this but FFmpeg simply copies the audio from the stream so the issue is likely your camera. What you can try is to set audio_codec under recorder to something like aac which will make FFmpeg re-encode the audio and maybe fix the pts issue

ceewanna commented 1 year ago

Every camera working now is aac audio and I have tried your suggestion and it made no difference.

I just found an issue where it discussed audio and video pts not synchronized. I am no expert on any subject here but what I can imagine is the synchronization if any will be at the time it created or it played. The other source of non-synchronization could be at the time the system received audio and video. However, when I put the rtsp on real-time stream I don't see any issue. Still this is a puzzle to me.

https://github.com/mpv-player/mpv/issues/9217

From this I tried playing some vdo files with vlc. It didn't seem to encounter such issue.

roflcoopter commented 1 year ago

Both use_wallclock_as_timestamps and avoid_negative_ts are used in the default FFmpeg command when saving segments to disk.

When a recording is finished, Viseron concatenates these segments into a single mp4. Maybe the problem occurs during the concatenation. What you can do is to play around with the concatenation command by setting output_args under recorder.