Audio corruption

[benroeder]

Symptom

UV will generate gaps and glitches (unflow and overflow) in audio wav files. see attached pictures Full track with transients marked

glitch in sine wave

Gap in audio

How to recreate

I have created this on a mac mini running 12.2.1 (M1) and on Macbook Pro 9 (intel)

M1

Decklink 8KPro output
Decklink mini recorder 4K
Blackmagic drivers 12.2.2a

UV version

 /Applications/uv-qt.app/Contents/MacOS/uv
MasterPortDestroyer is armed
UltraGrid 1.7+ (tags/continuous rev ef0806da built Feb  9 2022 11:24:25)    
Display device   : none
Capture device   : none
Audio capture    : none
Audio playback   : none
MTU              : 9000 B
Video compression: none
Audio codec      : PCM
Network protocol : UltraGrid RTP
Audio FEC        : none
Video FEC        : none

Display initialized-none
Video capture initialized-none
Created new RTP session with SSRC 0x675907f2.
Exit
MasterPortDestroyer triggered

output looped to input

cli

/Applications/uv-qt.app/Contents/MacOS/uv -t testcard:1920:1080:24:v210:pattern=bars:apattern=sine -d decklink:device=0 -r embedded -s embedded -VV --audio-capture-channels 2  > testsignal.log 2>&1

record with MediaExpress from DeskTopVideo

examine in Davinci Resolve Fairlight package

see wav m1 via wetransfer

see logfile via wetransfer

grep flow testsignal.log
[1645099798.618] [Decklink display] audio buffer underflow!
[1645100719.869] [Decklink display] audio buffer underflow!
[1645101391.862] [Decklink display] audio buffer underflow!
[1645103794.303] [Decklink display] audio buffer underflow!
[1645105038.296] [Decklink display] audio buffer underflow!

No mention of overflow which is what I would have thought created the "glitch"

Intel

UltraStudio Monitor 3G
UltraStudio Recorder 3G
Blackmagic drivers 12.2.2a

cli

/Applications/uv-qt.app/Contents/MacOS/uv -t testcard:1920:1080:24:v210:pattern=bars:apattern=sine -d decklink:device=1 -r embedded -s embedded -VV --audio-capture-channels 2  > testsignal.log 2>&1

same sort of issues

as @MartinPulec knows this happens under Linux as well.

The above is recreating the issue, but with a CESnet compiled app, no compilation or modification from me

[benroeder]

how to use resolve to find transients

Open resolve and click on the fairlight panel

click media pool

import media file

drag to timeline

increase size of track and click transient in both places

It will process for a while and then the following will appear

Then move cursor to transient line and zoom in to see issue

and

etc

[benroeder]

The using no low latency I also get errors

cli

/Applications/uv-qt.app/Contents/MacOS/uv -t testcard:1920:1080:24:v210:pattern=bars:apattern=sine -d decklink:device=1:no-low-latency -r embedded -s embedded -VV --audio-capture-channels 16  > testsignal.log 2>&1

Overview of recording

Audio gap

Audio glitch

wav recording file

test20220218.wav via wetransfer

log file

tessignal.log via wetransfer

grep flow testsignal.log [1645176074.300] [Decklink display] audio buffer underflow! [1645177441.160] [Decklink display] audio buffer underflow! [1645178731.806] [Decklink display] audio buffer underflow! [1645179573.744] [Decklink display] audio buffer underflow! [1645181171.479] [Decklink display] audio buffer underflow!

[benroeder]

To confirm issue is not with testcard, I generated a sine wave in Audacity

test sine wave file test file via wetransfer view in Resolve, note no transients

create folder with one video file and symlink to it many times to give use playback, and added above file

testplay.zip via wetransfer

cli

/Applications/uv-qt.app/Contents/MacOS/uv --playback ../testplay -d decklink:device=1 -r embedded -VV > testsignal.log 2>&1

record using MediaExpress

Overview, audio stops after a while, before end ?? different bug ?

zoom in

log

out log file via wetransfer grep flow testsignal.log [1645198473.750] [Decklink display] audio buffer underflow! [1645198473.791] [Decklink display] audio buffer underflow! [1645198552.645] [Decklink display] audio buffer underflow! [1645198563.491] [Decklink display] audio buffer underflow!

[MartinPulec]

Hi @benroeder, from my point of view, it is indeed given by the drifting clock of DeckLink device. As for the overflow, have you been able to test code from this branch? IIRC, the usage was -d decklink:drift_fix. But it is now able to handle only overflows, not underflows. The latter would be possible to add.

Also, I've added crescendo audio pattern. Usage is simply:

uv -s testcard:crescendo; or better
uv -s testcard:crescendo=4

(4 means 4x faster volume increase). The idea behind this that if you are able to capture output signal, the pattern better exhibits what happens - eg. in case of gap if the audio frames are dropped (thus the sound would be non-continuous when removing the gap) or there is clock shift (in this case it would be continuous).

[benroeder]

Hi @MartinPulec, I get the following compilation error on that branch, have not had a chance to look at properly yet ` make mkdir -p src/audio/ g++ -g -O2 -I. -std=gnu++17 -fPIC -msse4.1 -pipe -W -Wcast-qual -Wcast-align -Wmissing-declarations -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -I/usr/include/x86_64-linux-gnu -I/usr/include/opencv -I/usr/local/include -I/usr/include/opencv -I/usr/local/include -I/usr/local/include -I/usr/local/include -I/usr/local/include -I./ext-deps/zfec/zfec -fopenmp -DMAGICKCORE_HDRI_ENABLE=0 -DMAGICKCORE_QUANTUM_DEPTH=16 -fopenmp -DMAGICKCORE_HDRI_ENABLE=0 -DMAGICKCORE_QUANTUM_DEPTH=16 -I/usr/include/x86_64-linux-gnu//ImageMagick-6 -I/usr/include/ImageMagick-6 -I/usr/include/x86_64-linux-gnu//ImageMagick-6 -I/usr/include/ImageMagick-6 -g -DHAVE_CONFIG_H -DPATH_PREFIX='"/usr/local"' -DLIB_DIR='"/usr/local/lib"' -Wall -Wextra -Wpointer-arith -DGF_BITS=16 -D_GNU_SOURCE -Isrc -I. -I./src -I./test -Idxt_compress "-I/usr/local/cuda/include" -I./ext-deps/DeckLink/Linux -I/usr/include/SDL2 -I./glm -Icineform-sdk/Common/ -MD -c src/audio/types.cpp -o src/audio/types.o In file included from src/audio/types.cpp:47:0: src/audio/types.cpp: In member function ‘std::tuple<bool, audio_frame2> audio_frame2::resample_fake(audio_frame2_resampler&, int, int)’: src/audio/types.cpp:422:32: error: ‘class audio_frame2_resampler’ has no member named ‘resample_to’; did you mean ‘resampler’? UNUSED(resampler_state.resample_to); ^ src/debug.h:51:20: note: in definition of macro ‘UNUSED’

define UNUSED(x) (x=x)

                ^

src/audio/types.cpp:422:32: error: ‘class audio_frame2_resampler’ has no member named ‘resample_to’; did you mean ‘resampler’? UNUSED(resampler_state.resample_to); ^ src/debug.h:51:22: note: in definition of macro ‘UNUSED’

define UNUSED(x) (x=x)

                  ^

Makefile:279: recipe for target 'src/audio/types.o' failed make: *** [src/audio/types.o] Error 1`

[MartinPulec]

I see, it should be fixed now - it was just a compatibility if SpeexDSP was not found. I've fixed it in place and rebased against current master branch. However, compiling the branch without SpeexDSP would be useless, since the speex resampler is used to adjust the sample rate.

[benroeder]

Hi @MartinPulec, I ran the above

and zoomed in

I see gaps like

and

and

from the log I see

grep flow crescendo.log [1646927662.327] [Decklink display] audio buffer underflow! [1646927662.407] [Decklink display] audio buffer underflow! [1646927662.647] [Decklink display] audio buffer underflow! [1646927663.007] [Decklink display] audio buffer underflow! [1646927666.329] [Decklink display] audio buffer underflow! [1646927840.331] [Decklink display] audio buffer underflow! [1646928064.007] [Decklink display] audio buffer underflow! [1646928640.331] [Decklink display] audio buffer underflow! [1646929928.333] [Decklink display] audio buffer underflow!

log file crescendo.log

wav file extracted from recorded mov with

ffmpeg -i crescendo.mov -vn -acodec copy crescendo.wav

wavfile via wetransfer

cli used to generate

~/uv/UltraGrid/bin/uv -t testcard:1920:1080:24:v210 -s testcard:crescendo=4 -d decklink:device=4:drift_fix -V -r embedded > ~/crescendo.log 2>&1

[MartinPulec]

I've a bit walked through the data but as far as I understand it, it looks to be doing what is supposed to do, right? It just requests some time to settle the correct rate - it is gradually resampling to 47061, 4798, 47991, 47996 and 47998,8 Hz.

The problem may be the convergence - it overshoots the drift by a great deal at first... Also the final frequency should be closer to 48 kHz (I am deducing from Sending resample request -2: 12287694/256. The request -2 actually means that the computed drift is 1/128 Hz lower - this is just an aggressive step to avoid overrun, thus it should be rather something like 47999,3.

I don't have this tested, but what if you used --param soft-resample 12287822/256 instead of :drift_fix DeckLink option?

[benroeder]

@MartinPulec we have a pull request coming with a full fix for this issue.

[benroeder]

See pull request

[benroeder]

fixed in decklink resdampling