[MU4 Enhancement] Add option to choose the correct sound automatically when switching to Muse Sounds

[cbjeukendrup]

When you want to switch from Muse Sounds to MS Basic, that can be done with 3 clicks:

1. The triangle to open the sound selection menu
2. SoundFonts
3. MS Basic

The correct sound within MS Basic is chosen automatically.

But when you want to switch from MS Basic to Muse Sounds, it's a bit more work:

1. The triangle
2. Muse Sounds
3. The library you want
4. The instrument you want

It would be very nice if the correct sound within Muse Sounds could be selected automatically. I see a couple of possible options:

• Just clicking the Muse Sounds item directly in the sound menu (so, clicking the item would automatically select the correct sound, and hovering the item would open the submenu) (possible disadvantage: accessibility)
• Inside the Muse Sounds submenu, an item called "Auto"
• Inside the Muse Sounds submenu, an item called "[the name of the instrument that you would get by default]"

[DmitryArefiev]

@avvvvve @bkunda So what about to do the same as we did with MS Basic?

(and actually it will save some time when testing switching sounds in Mixer :))

[avvvvve]

@DmitryArefiev @bkunda

That's a great solution to me—always a fan of keeping things 1:1 if we already have an established pattern that works!

To be overly clear, it would go in the second layer of the context menu, after hovering over Muse Sounds like so:

[DmitryArefiev]

@avvvvve The only thing is left, what we should do if mapping with MuseSounds doesn't work..? (Expected result)

[shoogle]

We could disable (grey out) the "Choose automatically" option if the instrument that would be chosen isn't installed. This could happen if the user only has some (not all) of the Muse Sounds libraries installed, or if Muse Sounds doesn't have a sound for that particular instrument yet.

[avvvvve]

Task description

As a user choosing sounds in the mixer, I should be able to select an option that chooses the correct Muse Sounds sound for the current instrument so I don't have to look for the correct instrument in the context menu layers.

Requirements

• Add "Choose automatically" and a divider line to the top of the context menu that appears when you click a Mixer channel's sound button and hover on Muse Sounds (copy how this appears in the MS Basic context menu).

• When clicked, "Choose automatically" becomes checked. In the background, MS assigns the corresponding Muse Sounds sound (i.e. if instrument is "Trumpet in B♭", clicking "Choose automatically" will assign the Muse Sounds "B♭ Trumpet" sound).

• "Choose automatically" is enabled only if the user has already installed the corresponding Muse Sounds sound for the instrument. For example, if Muse Strings isn't installed, the "Choose automatically" will be disabled for a Violin instrument. Disabled style: 30% opacity

• When Muse Sounds is activated as the playback profile in Playback setup, "Choose automatically" should become selected for all instruments. If there is no Muse Sounds instrument, select "Choose automatically" within the MS Basic menu.

[avvvvve]

Looking deeper into soundfont selection and I just realized something we may want to consider.

My (incorrect) assumption: clicking "Choose automatically" in the MS Basic flyout finds and selects the soundfont instrument that corresponds to the channel's instrument. Wrong—"Choose automatically" is not an action, but a selectable option itself.

What's important is that the behavior that results from selecting "Choose automatically" is different from selecting the actual instrument from the submenus. For example, say I have a trumpet part with a "mute" element, which changes the sound that is played back in the score. Within MS Basic, there is a regular Trumpet sound and a Harmon Mute Trumpet sound. "Choose automatically", as far as I can tell, decides which of these to play based on the presence of the "mute" element. If I instead specify regular MS Basic Trumpet as the sound in the mixer, "mute" no longer plays back the muted trumpet sound.

So here, "Choose automatically" means "switch the sound to the correct playback sound based on score elements that affect playback".

I need to think more about whether this behavior is desired in relation to the Sound Flags designs, but my hunch is that sound switching should be handled only by sound flags, and "Choose automatically" should no longer be a setting, but an action.

[shoogle]

my hunch is that sound switching should be handled only by sound flags

Agreed. If I switch between the "Violins 1" and "Violins 2" sounds then I would still want to have pizz. and tremolo playback.

"Choose automatically" should no longer be a setting, but an action

Maybe, but not necessarily. If it's a setting then that implies the sound could change later on, which might be good or bad depending on the situation.

For example, if your score contains a Clarinet in A but we only have sounds for Clarinet in Bb and Clarinet in Eb, if you pick one of those sounds you'd be stuck with it even if we add a proper Clarinet in A sound later on. "Choose automatically" could be useful as a way to opt-in to getting a better sound in the future.

Interestingly, if you add a Clarinet in A to your score now, it uses MS Basic rather than the Bb or Eb clarinet from Muse Sounds. We'd probably rather it used the Muse Sound, but regardless, let's say Muse Sounds does add a Clarinet in A at some point. What would we like to happen to older scores that were using MS Basic and "Choose automatically"?

[avvvvve]

I just recorded a quick walkthrough of my above message for anyone curious to see how playback behavior is affected by "Choose automatically."

Peter, all good points and I agree that setting does actually make sense. I'll revise my spec above to align the "Choose automatically" behavior for Muse Sounds with the current behavior for MS Basic.

https://github.com/musescore/MuseScore/assets/20806406/5cb8c7f4-34e6-4944-b8fc-e35080f03dfd

[bkunda]

Ok so if I were to summarise what I'm seeing here:

1. It is desirable to have "choose automatically" as a submenu option for Muse Sounds instruments in the mixer. This solves the following problem: when an instrument doesn't automatically trigger a Muse Sounds selection in the mixer, the user currently has to navigate through the entire Muse Sounds library to find the most appropriate instrument; selecting "choose automatically" leaves the decision up to MuseScore, reducing cognitive load on the user.
2. Text instructions that affect playback, like "mute", don't currently work when a specific instrument is selected in a Soundfont, but they do work if "choose automatically" is set for that Soundfont. We can resolve this problem by converting these text elements to sound flags (v.4.3).
3. Some of these text instructions don't work when a MuseSounds instrument is selected: this is usually because we haven't mapped this element to the right sample (it may be something we have to do on our end). When these issues are found, it would be good to report them and add the muse sounds label.

Let me know if I've missed anything important here.

@avvvvve Could you please raise this as a separate issue/implementation task, citing that it would resolve the present issue (It'll just make the task more obvious to whoever implements this) 🙂

[avvvvve]

That's it! Will do.

I might add on a slight tweak to this: It would be nice if, when "Choose automatically" is selected, there was some visual indicator as to which sound it has chosen. Could be as simple as a dot in place of the checkbox. That way, if there isn't a proper sound to map to the instrument, the lack of such an indicator would at least signal that something is off.

@bkunda let me know what you think!

[shoogle]

@avvvvve, I guess the arguments I made about not automatically switching to Muse Sounds are equally applicable to the "Choose automatically" option. Namely, if I select "Choose automatically" and get one instrument sound today, but then it gives me a different instrument sound tomorrow, the balance might be off or notes might be out of range of the new sound, and I might not notice before it's too late to do anything about it.

So perhaps you were right originally when you said "Choose automatically" should be an action rather than a setting.

However, if want to show notifications like this in the future:

A new Clarinet in A sound is available. Switch to it now? [ Yes ] [ Cancel ]

Then we would need to store something like this in the score file now:

instrument: Clarinet in A
sound: Clarinet in Bb
sound-chosen-automatically: true

In this case, "Choose automatically" is both an action and a setting. If we add a proper Clarinet in A sound later on then the user would see the notification that a better sound is available, but we wouldn't switch to it unless the user asks us to.

[avvvvve]

@shoogle I might be misunderstanding, but I don't think this is possible:

Namely, if I select "Choose automatically" and get one instrument sound today, but then it gives me a different instrument sound tomorrow...

And that's because... There are separate "Choose automatically" options for Muse Sounds and MS Basic, so no auto-switching between the two will happen.

Say I don't have Muse Strings installed, and Muse Sounds is my activate playback profile. A violin instrument will "fall back" on MS Basic - and "Choose automatically" within the MS Basic menu will be selected*. In the Muse Sounds menu for the violin instrument, "Choose automatically" will be disabled because the sound is not available.

Later, I install Muse Strings. Violin is still set to MS Basic's "Choose automatically". But the "Choose automatically" in Muse Sounds is now active. I can also reset the Muse Sounds profile to load in the new Muse Strings violin sound. But we're still not going to auto-switch a new sound in.

*right now, nothing actually is visually "checked" in the MS Basic submenu if the sound hasn't been touched, but I believe it behaves as if "Choose automatically" were checked.

[shoogle]

@avvvvve, it would happen if we add a new sound to MS Basic. In that situation MS Basic's "Choose automatically" option for one or more instruments would change to use the new sound instead of whatever they currently use.

You're right that it wouldn't happen with Muse Sounds. Right now we only apply Muse Sounds automatically when the instrument and sound are an exact match. If your instrument is Clarinet in A then you'll get MS Basic because Muse Sounds only has sounds for the Bb and Eb clarinets. (Arguably we should fall back to the the Bb or Eb sound instead of MS Basic, but right now we don't.)

But if you think about it, the only reason to make "Choose automatically" a setting is to enable it to give you a different sound in the future. If we are certain that it will always give you the exact same sound then we might as well pick that sound now (i.e. make "Choose automatically" an action rather than a setting) because we know that it will never ever change.

[avvvvve]

the only reason to make "Choose automatically" a setting is to enable it to give you a different sound in the future

Not the only reason—it also makes elements like "mute" and "open" affect playback in MS Basic Trumpet (these do not work if you don't have "Choose automatically" selected). See my screen recording here. This will no longer be the case, however, when we implement sound flags.

I think what this conversation is getting at is that it's not clear what "Choose automatically" does as a setting. I'd still argue that we should never automatically change a sound in a user's file just because they installed something new—we should always prompt them first—so if "Choose automatically" as a setting implies that this would happen, let's revisit it being an action.

@jessjwilliamson, since I am butting in to this without knowing the considerations that went into the soundfont design, are there any other reasons why "Choose automatically" needs to be a setting rather than an action?