Multichannel-Recording gets out-of-sync over time (2)

[hacst]

Followup to #964 reported by @rtrind :

I am the "technical guy" for a small podcast in Brazil and since we discovered mumble it was a pleasure using the record feature to skip using Skype audio and individual local recordings.

But we are having serious desync issues all the time. We were using a 1.3.0 snapshot version on a local server at my house. After 20 minutes of recording, the audio tracks start to drift and they continue to drift until the end of the tracks.

We all use continuous mode all the time because there is always a popping sound on the beginning of each voice capture (not always, but frequent enough we notice it, we will open another ticket for this after the recording issue is solved). Some users were having audio aggressive delays during the recordings (because of the low quality brazilian internet, reaching 4000 latency), so we put the murmur server on the Amazon EC2, Sao Paulo (very low latency, 15~20ms). This solved most of the delays while talking with 4~6 clients in continuous mode.

Even so, the recording does not reflect the audio we hear while live, the tracks are still drifting. One user was recording on wav format and me on flac. Both representations have the drift.

Right now the murmur server is using version 1.3.0~540~gedd5509 on a Ubuntu 14.04.2 LTS, hosted on Amazon EC2. The live session goes perfect, without any problems (except for some specific problems with some users), but the recording always suffers. We believe most of the clients are up to date, but we never checked one by one.

I don't know what you guys need us to deliver for this analysis to be performed. Feel free to ask anything and I will help as best as I can.

The recorders are myself and the podcast editor. Both always update to the latest snapshot when prompted by the software, so we are always using the latest one. Last recording we were at the same gedd5509 version on both clients. Some other participants in the call could be using other clients, we didn't checked.

[hacst]

I think the continuous transmission might have something to do with this. The way the code is written - assuming it isn't buggy in some other way - should only allow a 100ms desync between the streams. The realignment is done by inserting silence into streams that are falling behind (be it due to silence or other reasons) compared to a local reference against a high resolution time source (disconnected from actual output). No separate compensation is done for receiving too many samples though. In theory a positive sample rate drift of a few percent would add up over the recording.

@rtrind : Is it always the same peoples streams that pull ahead? Do the streams realign if you stop all transmission for a while before continuing to speak (e.g. at the end of a recording)?

[rtrind]

I am not really sure, since I only have the last recording at hand (I deleted the original recordings done on the local server, kept only the last one from Amazon).

The editor says it's variable but do not take this for granted, it's just an impression over the last 3 recordings, on different servers and with some users with latency sometimes, but not always.

In this last one, all the tracks have different sizes and are out of sync pretty early (70 minutes recording, looks out of sync at 5 minutes mark). When I put them on a editor and change the tempo so all of them have the same size, they are pretty much in sync, except for one of the tracks, who after a 10 minute silence in minute 40 gets completely de-synced. Inserting a manual silence there makes it good enough again.

I am scheduling a special recording session just to troubleshoot this and we will record again in the same fashion, but every 5 minutes we will make a syncing sound so later we can quickly identify where are the desyncs and make the test more objective.

We could record in the future with voice activity mode, but 20~30% of the time, when we start talking the sound pops or becomes robotic for a micro second. It's not critical (could be adjusted by the editor), but it affects the quality, so we would like to solve this using the continuous mode if possible.

If you have any tips for the special recording session, like using a specific recording format or using some special settings, please let me know so we can share more information as possible.

Thanks!

[ubuntuaddicted]

in reference to the audio popping at the beginning of speaking after silence, we have this occur as well on our podcast. it's totally random and we can't pin point the issue. it's very annoying since it occurs after silence. we all have headphones so it's almost defeaning it's so loud. we all use voice activity/amplitude because it's a podcast and don't want to have to worry abvout using push to talk while doing our podcast. any info related to this or how to fix would be great. we're using 1.2.8.1~ppa~trusty1

On Sat, Mar 28, 2015 at 2:59 PM, rtrind notifications@github.com wrote:

I am not really sure, since I only have the last recording at hand (I deleted the original recordings done on the local server, kept only the last one from Amazon).

The editor says it's variable but do not take this for granted, it's just an impression over the last 3 recordings, on different servers and with some users with latency sometimes, but not always.

In this last one, all the tracks have different sizes and are out of sync pretty early (70 minutes recording, looks out of sync at 5 minutes mark). When I put them on a editor and change the tempo so all of them have the same size, they are pretty much in sync, except for one of the tracks, who after a 10 minute silence in minute 40 gets completely de-synced. Inserting a manual silence there makes it good enough again.

I am scheduling a special recording session just to troubleshoot this and we will record again in the same fashion, but every 5 minutes we will make a syncing sound so later we can quickly identify where are the desyncs and make the test more objective.

We could record in the future with voice activity mode, but 20~30% of the time, when we start talking the sound pops or becomes robotic for a micro second. It's not critical (could be adjusted by the editor), but it affects the quality, so we would like to solve this using the continuous mode if possible.

If you have any tips for the special recording session, like using a specific recording format or using some special settings, please let me know so we can share more information as possible.

Thanks!

— Reply to this email directly or view it on GitHub https://github.com/mumble-voip/mumble/issues/1617#issuecomment-87293018.

My Youtube Channel http://www.youtube.com/user/ubuntuaddicted has tech, gaming and comedy all in one. My Tech Blog http://ubuntuaddicted.blogspot.com Our Linux Gaming Website http://linuxtechandgaming.com and Podcast

[rtrind]

You were right, the problem is probably related to the continuous mode. We recorded a 90 minutes podcast and no problems on syncing. Any ideas on what could this be and the next steps?

PS.: I will probably soon open another issue for the popping on the beginning of voice activity.

[Zylann]

I just had sync issues as well when recording 4 hours with multichannel enabled. It resulted in 5 files with different lengths (random 1..10 seconds diff). I tried to stretch them so they get the same duration in an audio editor, but voices are still out of sync. It's like individual audio files got random, unnoticeable drops over time.

[radoslawg]

Same here. However, I run 1.2.13 client - I will check 1.3.x snapshot this friday. Recording to ogg, 5 clients, one of them on continuous mode, recording time over 4 hours. Every file has different duration (+/-5 seconds) and seems to drift at different pace or "looses" time at random places.

[itsrachelfish]

Looking forward to seeing this resolved! Big issue for podcast producers.

[reelsense]

http://rogueamoeba.com/audiohijackpro/

Record outside of mumble and save the multi track files for inserting edits. Mumble uses separate timers for every track for some reason... So this is very much an expected side-effect.

[ghost]

http://rogueamoeba.com/audiohijackpro/

Record outside of mumble and save the multi track files for inserting edits. Mumble uses separate timers for every track for some reason... So this is very much an expected side-effect.

This seems like a valid solution. Or something similar like Audacity. Most podcasts will probably want some level of editing and other post-production, anyway. Mumble isn't really built for that. If other members can weigh in on if this is/was an issue we want to "fix."

[Krzmbrzl]

I believe this would indeed be something we want to fix. However, I believe there have been some fixes with respect to audio recording desync, so the question really is: does this still happen?

[github-actions[bot]]

This issue has been marked as stale, because our request for more information has thus far not been fulfilled.

If no further action occurs, this issue will be closed within 7 days.

[z411]

I believe this would indeed be something we want to fix. However, I believe there have been some fixes with respect to audio recording desync, so the question really is: does this still happen?

@Krzmbrzl I'm not sure about recording but continuous mode does desync after some time if the connection is not perfectly stable. Using on 1.5.517, drifts faster when using higher bitrate.

The way to work around it is to mute and unmute the person; after muting the remaining audio in buffer plays in its entirety before muting, so if the audio was desynced 5 seconds, muting will work 5 seconds later. But it will start drifting again and it has to be done again.

The person with the unstable connection can hear the audio drifting with the server loopback test.

If it's the same issue mentioned by OP, then it's an issue with Mumble itself that gets noticeable with continuous mode, and people just happen to notice it even more when recording. If there's any stats or info I can provide I'd be happy to help debug the issue.