[Feature request] Microphone input delay calibration

[deuteragenie]

The microphone delay setting is currently located in the general options. Intuitively, I was searching for it under the "Record" tab, where other input related options are located.

Also, it would be great if it was possible to calibrate this delay somehow.

[hackerb9]

It would be nice if one could simply hold the microphone up to the speakers while UltrastarDX plays some beeps and boops to have it calibrate itself.

[HermannDppes]

While this might work well to calibrate the delay between audio input and audio output, it ignores the calibration of the video output. To judge whether this issue is relevant, I see two important questions:

• Is there a lot of variation between audio-out delay and video-out delay? If not, they can easily be configured as one.
• Do people use the video for (precise) timings? I think I personally do, but I am not sure. If practically noone does this, it does not warrant a calibration feature, I guess.

[s09bQ5]

TVs nowadays do some image processing that can delay the video significantly. They usually have a special mode for games that need a low delay. I think it would be confusing if the wrong syllable is emphasized while you try to sing. But if the delay is less than 100ms, I usually don't notice.

For automatically calibrating the audio/video delay, one could build a little device that plays the output of a photo diode controlled VCO on a small speaker.

[basisbit]

This issue is about audio output + microphone input delay. For display delay, about the only thing you could offer is a setting to increase / decrease a delay value by how much the display is offset from audio. Regarding audio delay and synchronisation, the typical use case would be someone playing USDX who uses a bluetooth connected speaker with high audio output delay because of buffering - fir example Amazon Echo with > half a second of delay.

[logon84]

This issue is about audio output + microphone input delay. For display delay, about the only thing you could offer is a setting to increase / decrease a delay value by how much the display is offset from audio. Regarding audio delay and synchronisation, the typical use case would be someone playing USDX who uses a bluetooth connected speaker with high audio output delay because of buffering - fir example Amazon Echo with > half a second of delay.

I was going to add a feature request, because last weekend I was trying to sing with a bluetooth speaker and that is what happened, the sound came after the notes....I had to edit the GAP for every song I wanted and add 300ms. Maybe it will be a good idea to add an option like "delay every song X ms".

[s09bQ5]

Maybe it will be a good idea to add an option like "delay every song X ms".

What's wrong with the current options "Audio/Video Delay" and "Microphone Delay"? I added the latter under "Game" instead of "Record" because there is no space in the "Record" menu and the GUI framework is not flexible enough to allow for scrolling.

With the PortAudio and SDL backends it is also possible to suggest a value for the buffer size individually for each recording device by setting the Latency[x] value in the config.ini file. For PortAudio this is the size of the whole buffer in milliseconds. For SDL this is the size of the chunks received from the sound card in milliseconds (i.e. the period size in ALSA speak). I don't know why we fix this value to 20ms for BASS.

[hackerb9]

What's wrong with the current options "Audio/Video Delay" and "Microphone Delay"?

This request is not about A/V delay. The microphone delay is not located in an intuitive place and it does not have calibration.

[dgruss]

I was wondering why the lag has gone up that much compared to the CMD version. There I typically had a mic delay of 40ms and now it's 290ms on the exact same setup. 290ms is high enough to cause a lot of irritation while playing as the feedback lags so much behind. That's on a fully updated Windows 10 where the CMD version only works in window mode (fullscreen crashes).

[s09bQ5]

Just stumbled upon jack_delay and am astonished by the simplicity of its code. Maybe we can use the algorithm to automate the calibration.

[TheNotary]

I was wondering why some songs seemed to have the notes come on too soon, whereas others had them coming on way too late. Everyone transcribing songs is probably not using an environment that is calibrated the same way. Maybe there should be a separate button for calibration so people rummaging through the settings can clearly get to the feature. I had assumed there wasn't any feature around setting the timings available when I finished setting up a mic.

[lesderid]

I was wondering why some songs seemed to have the notes come on too soon, whereas others had them coming on way too late.

Actually I think it might be because of different audio encodings etc. too (e.g. max compression FLAC at 192kHZ 32-bit 7.1 is probably slower to decode/resample/downmix than 44.1kHZ 16-bit uncompressed PCM), but I haven't confirmed this.

[s09bQ5]

Actually I think it might be because of different audio encodings etc. too (e.g. max compression FLAC at 192kHZ 32-bit 7.1 is probably slower to decode/resample/downmix than 44.1kHZ 16-bit uncompressed PCM), but I haven't confirmed this.

No, if that was the case, we would have buffer underruns every time a song is started.

Resampling might introduce a tiny delay, though, if the peak of the impulse response of the low pass filter is not at its start. Usually the IR of a low pass filter is symmetric with its peak in the middle. If we knew the delay introduced by a filter, we could skip the first few samples. But for real time resampling the IR is probably so short that it doesn't matter.

[dgruss]

I just discussed this yesterday while playing around with some software devs: The delay on different hardware is different (challenging if you use different types of microphones, different soundcards, etc. simultaneously), different Windows versions, Linux vs. Windows, USDX versions, PC <-> speaker connection (HDMI, 3.5mm, USB audio interface, HDMI directly to the AV receiver or tunneled through the TV..., bluetooth speakers, etc.) there seem to be many influence factors, I would guess too many to control.

A simple solution would be an automatic input calibration: In principle you could, on some settings page, just play a constant sequence of audio alternating between 2 frequencies (every 1000ms or so). When the microphone picks up the frequency change, that is the same delay you would see with someone singing perfectly in sync. That's the delay you need for that specific microphone.

(The audio-video delay should be considered a separate issue and it can be solved separately. This is purely about the mic delay.)

[lesderid]

@s09bQ5 Makes sense, thanks for clarifying!

@dgruss Yes, that's the main point of this issue I believe.

[s09bQ5]

A simple solution would be an automatic input calibration: In principle you could, on some settings page, just play a constant sequence of audio alternating between 2 frequencies (every 1000ms or so). When the microphone picks up the frequency change, that is the same delay you would see with someone singing perfectly in sync. That's the delay you need for that specific microphone.

That's what I did when I made #450. I made a song that alternated between two frequencies and held my headphones to the microphone. I then adjusted the delay until the recorded frequency transitions matched the timing in the .txt file.

A fully automatic and more precise method can be found in Fons Adriaensen's jack_delay tool.

Btw., the delay introduced by converting 44.1 kHz audio to 48 kHz audio with SDL2 is 1 sample (0.02 ms).

[basisbit]

A fully automatic and more precise method can also be found in UltraStar Play. We implemented it there a few years ago and it is MIT licensed.

[lesderid]

I'm willing to pay a 50 EUR (~55 USD) bounty for a good (jack_delay style?) implementation that gets merged. (I can use PayPal, bank transfer, or a donation to a charity of your choice. Offer expires 6 months from now.)

[hackerb9]

Looks like the calibration source code from Ultrastar Play is here: https://github.com/UltraStar-Deluxe/Play/blob/master/UltraStar%20Play/Assets/Scenes/Options/RecordingOptions/CalibrateMicDelayControl.cs .

[lesderid]

I'm willing to pay a 50 EUR (~55 USD) bounty for a good (jack_delay style?) implementation that gets merged. (I can use PayPal, bank transfer, or a donation to a charity of your choice. Offer expires 6 months from now.)

I'm increasing this to 100 EUR (willing to pay 50% upfront), offer lasts till end of year.