[MU4 Issue] Soundplayback after note input noticeably delayed

[ChrisMuellersCode]

Describe the bug When entering a note - regardless of the input method and regardless of whether or not note input is enabled - the soundplayback is noticeably (~0,5secs) delayed. This makes trying out a line (or sth.) before writing it down impossible. This forces some users - including me - to continue using MU3 (see forums).

I've already tried a view things to narrow it down. It seems to be a general issue however. For details please see "Additional context" section below.

To Reproduce Steps to reproduce the behavior:

1. Open any score or create a new one.
2. Select a staff.
3. Click on a note on the screen piano or press a key on your MIDI Keyboard.
4. Notice the delay between the click / keypress and the playback

Expected behavior When pressing a key on the MIDI keyboard or the on screen piano, the playback of the sound is near instantaneous (at or below around 20ms with a hardware like mine).

Screenshots

• None

Platform information

• OS: Windows 10 Home, 21H2, OS Build: 19044.2486
• CPU: 12th Gen Intel(R) Core(TM) i9-12900K 3.20 GHz
• Ram: 64GB
• SSD for OS and Musescore: Samsung SSD 980 PRO 2TB
• MIDI Keyboard: M-Audio Oxygen Pro Mini
• Audio: Focusrite Scarlett 2i4 (1st gen)
• GPU (for what its worth): 2047MB NVIDIA GeForce RTX 3080 (Palit Microsystems)

Additional context Below is a List of things I found out while trying to solve the issue on my own. I've tried all that with the current Release Version of MU4 (4.0.1) and the latest nightly build (MuseScoreNightly-230280506-4.0.2-cfffaa1-x86_64). Behavior is the same in both versions.

• The lagging is not there in MU3 (parallel install on same system).
• The lagging occurs always, regardless of sound selection in the mixer (MS Basic and Muse Sounds).
• The lagging occurs on both, my PCs internal soundcard and my external Focusrite USB Soundcard.
• Changing the buffer size on either soundcard does not change the lagging.
• It does not matter, whether the note is entered via one of my midi keyboards, the screen piano or by clicking somewhere on the stave in "note input mode".
• It also does not matter whether or not note input is enabled. The "note input state" that renders MU4 unusable as a writing tool for me, is the lag when note input is not enabled, of course.
• Score size does not seem to have an impact on the lagging. It feels the same (around half a second, as mentioned) in a simple one page piano score as in a large multiple page score like the Nicholas Dodd's - "Searching Four" score that is currently featured as inspiration in museHub.

Possible relations This issue may be related to the following: https://github.com/musescore/MuseScore/issues/15649 https://github.com/musescore/MuseScore/issues/15354 https://github.com/musescore/MuseScore/issues/13052

[ChrisMuellersCode]

Forgot to mention: If there is anything I can do to help solve this, please let me know!

[moonwave99]

Same here, everything fine with MS3 (parallel installation), huge lag with MS4 and now no sound at all regardless of the score size. Playback goes well with other audio programs (e.g. Logic).

M1 Macbook Pro OSX 12.6.3 MOTU Microbook II audio interface

[spaceo]

I have the same problem. Musescore 4 is useless to me because of the problem. Happy that I kept the previous version. I hope you will look into it ;)

OS: macOS 13.0, Arch.: x86_64, MuseScore version (64-bit): 4.0.1-230121751, revision: github-musescore-musescore-9b70a8c

[xcvii]

Same issue here on M2 Macbook Pro, macOS 13.2 MuseScore 4.0.1, MIDI input is delayed. MuseScore 3.6.2 it works perfectly with no noticeable latency. This issue forces me to keep using MS3, so very keen to have this fixed, let me know if there's anything I can help with.

[Jarrod-Herbert]

I am also experiencing this issue, on an M1 Macbook Pro. I think an important question to answer is whether the delay is in input - the time it takes for the message to go from Input -> MuseScore, or MuseScore -> Audio Engine

Perhaps that will help turn over the correct rocks. I unfortunately have no idea what I am doing!

[bkunda]

We did actually try to address this in https://github.com/musescore/MuseScore/pull/15010 (it was actually worse than it is now), however I agree that the problem still persists to a noticeable degree, and we should ultimately be aiming for no noticeable delay at all.

[schoutens]

I also want to strongly urge solving this issue: to bring it back to the way MU3 worked.

[Jojo-1000]

I measured the latency from the button press to different parts of the processing chain:

1. Piano press to midi event: 1-2ms
2. Midi event processed: 20-30ms
3. Synthesized note (MS Basic): 30-40ms
4. Samples read from audio buffer to system: 110-250ms

So the majority of the latency is from the audio buffering, which cannot really be shortened too much or there would be stutter (this depends on the number of instruments in the score, regardless whether they are playing). In the audio worker thread, there is a 2ms sleep between processing calls. If I remove that, there is a bit more room for a shorter buffer, but not a lot.

There is also a 20ms timer between receiving and processing the midi events, which could be shortened unless there is a particular reason for that delay.

[bkunda]

Thanks for looking into this @Jojo-1000! I was actually chatting about this issue earlier today with @RomanPudashkin. Rest assured, we are prioritising this for the 4.1 release.

[StephenCBryant]

I have had the same problem and have stuck with version 3. I asked Marc Sabatella and while responding to his questions and suggestions, made some progress:

1. does the note itself appear instantly or is that also slow? The note is visible right away, the sound lags behind [while using my Keystation 88 MK3 as an input device].

2. does it respond faster with normal computer keyboard or mouse input Typing on the computer's keyboard, say CDCDCDCDCDC, which I can do rapidly, the behavior is the same: the notes appear on the score immediately but the sound lags behind/

3. if you press a key and on your computer keyboard and hold it, do the notes enter quickly or is there lag between each? If I press and hold a key, it takes a little while to build up speed, and when I lift my finger, all the notes have been displayed but the last few play later.

4. is it equally slow when using Muse Sounds as when using MS Basic? [This refers back to the first three points.]

   1. both have some lag, but I'd say Muse Sounds/Grand Piano has more lag than MS Basic.
   2. same answer, Muse Sounds is a little laggier
   3. MS Basic shows and plays the notes faster than Muse Sounds, but also gets just a little behind. Perhaps it's because the notes are generated so quickly in this case, but the difference is more noticeable than in tests 1 and 2, that is MS Basic catches up more quickly.

5. does it get faster if you close the Properties panel? It might be a little faster with Properties closed, but still lags. It's a small difference; it feels a little faster but it's not obvious.

6. does it get faster if you change the buffer size to 1024 That does seem to make a difference - subtle, but in the right direction. [I had increased the buffer size to 4096 some time in the past]

But then I changed one other thing. When I looked at Preferences - I/O, I noticed that the Audio device was set to "System Default". I changed that to the actual name of the default device, and the lagginess was reduced quite a bit - but MS Basic is still more responsive.

So, with Properties displayed, the buffer size at 1024, and the Audio Device explicitly selected, the overall behavior is significantly better. MuseScore 3 is snappier, but I will now consider switching to the new version.

This is in Windows 11 on a 4-year-old laptop with plenty of RAM (16 GB). I'm also just working with Piano music and can't say how I'd fare with an orchestral score.

[jmickelin]

I measured the latency from the button press to different parts of the processing chain:

1. Piano press to midi event: 1-2ms

2. Midi event processed: 20-30ms

3. Synthesized note (MS Basic): 30-40ms

4. Samples read from audio buffer to system: 110-250ms

So the majority of the latency is from the audio buffering, which cannot really be shortened too much or there would be stutter (this depends on the number of instruments in the score, regardless whether they are playing).

While I don't know enough (any) of the internals to claim with any authority that this is doable, could it not be possible to reconfigure the buffer size to a smaller one in situations where only a single instrument is active? For example when playing on the virtual piano (with no other playback going on at the same time) or when a channel has been set to Solo in the mixer panel. To me, these are the two situations where low latency is most critical (when playing back a full score with multiple instruments, not as much), and given that it's almost a full order of magnitude larger than the other sources of delay, it does not sound too unreasonable to have a special optimization for this scenario.

[Apprisco]

So.... when is this being fixed? 4.4 now? In this thread, this issue was being prioritized for 4.1. This is an issue that's been here since 2.0. Like, please man.

[odkken]

Some more measurements vs buffer size:

My setup is garageband recording a loopback from another track synthesized by garageband, as well as the output of musescore. I built from source and added more buffer size options, all the way down to 32. The reported number is the time delta between the audio produced by garageband (always first) and the audio produced by musescore:

4096: ~440ms 2048: ~290ms 1024: ~210ms 512: ~160ms 256: ~140ms 128: ~120ms 64:~110ms 32:~110ms

Note: The baseline delta between garageband recognizing midi input and garageband synthesizing an audio sample is ~60ms. So all the numbers above should have 60ms tacked on to get an "absolute" delay from the time you press the key (practically speaking).

I'm using a Scarlett 2i4 on a mac pro M1.

For a ~2x improvement, simply reducing the value of MINIMUM_BUFFER_SIZE in audiotypes.h to 64 would do the trick. However, IMO 170ms is still an unacceptably distracting amount of latency. Something else needs to happen as well.

[bill0287]

I joined just so I could add more to the conversion. There is a lively discussion about this on the musecore forums. I see lag as well, here are my details:

Dell XPS 8960 Windows 11 Home 64-bit (23H2), OS Build 22631.3155, Intel 13th Gen i7-13700 NVidia GeForce RTX 4060 Ti 8GB Graphics 32GB RAM 1TB SSD system drive 7200 RPM internal recording drive (2TB) Musescore 4.2.1 Presonus Studio 1810c interface Interface settings:

• Sample rate 44.1KHz
• Block Size 256

Midi Keyboard used for testing: M-Audio Keystation 49es MKIII

Also Present:

Pro Tools (latest version) A handful of VSTs A few other minor apps, that's about it

Basic Sound fonts have almost no lag, but Musesounds and VSTs have lag.

Version 3.6 - zero lag, works amazingly

When using a VST through Pro Tools or Studio one there is no Lag at all. Midi keyboard works like a charm.

[scorster]

@ bill0287 commented

I see lag as well ... There is a lively discussion about this on the musecore forums.

I believe this may be that lively discussion.