Discussion on the current LMMS detuning implementation

This is copied from my post on the lmms discord for people who aren't on the discord

I think there should be a discussion about the LMMS note detuning implementation

Right now the note detuning implementation relies on the concept that each note has its own individual automation clip under the hood, when we detune the pitch of a note, it will just apply what we do to this specific note I'd say this approach works but the way it's done is kind of weird first of all by default this automation clip is empty, I feel like it should by default have two points that delimit the length of the note, there shouldn't be the ability to have the pitch detuning go over the length of the note as it will never be played, or at least it should not appear in the midi editor, right now it does, it should probably cut the remaining part that is outside the bounds of the note and should always have two points that define the start and end of the note

also, the detuning is done in a separate window, it works but it makes it harder to do precise modifications, it might be better to have the ability to directly edit it on the midi editor talking about the editor itself, right now it lacks a lot of options, it seems like it's not possible to set a different interpolation type per point and instead we can only set an interpolation type for the entire detuning of the note, and the pitch curve appearing in the piano roll only displays linear interpolation rather than the actually chosen one in the detuning popup

I feel like it would be good to discuss a bit about how it could be improved as the detuning will become important for the UTAU integration and could be very useful for various instruments such as synths etc there's also the fact not all lmms instruments actually support the note detuning in a way for this I feel like lmms's core features should be made in a way where they're sure to work on all built in instruments

I'm right now thinking about adding support for stuff like portamento, vibrato etc but that's why I kind of want to discuss about the way detuning in lmms works in general because it will make it easier to define the way to go about extending it

as an example, one way to do things that I like having worked with OpenUTAU is this OpenUTAU is really visual in its way to handle note detuning, and makes it easy to manually handle the pitch bending between two different notes

https://user-images.githubusercontent.com/16335601/141665099-2b19a9dd-c7fe-4d74-8243-35ff0ef0e8fe.mp4

Copying over some things mentioned on Discord + my own thoughts. To put my summary upfront, I think this should be replaced by two or three smaller issues:

Note detune should open with a point at the start and end of the note
Automations should support waves (or more generally repeating patterns) between two points
Automations should be editable inline (similar to #5541, so maybe they could be consolidated)

Invalid or duplicate

There shouldn't be the ability to have the pitch detuning go over the length of the note as it will never be played

False due to sustain/release.

Not all lmms instruments actually support the note detuning

Duplicate of #1121

The pitch curve appearing in the piano roll only displays linear interpolation rather than the actually chosen one in the detuning popup

Valid bug, but there's an existing PR for this: #5928

Automation editor issue

[It's] not possible to set a different interpolation type per point

Should be fixed in the automation editor, some work towards this was already done in https://github.com/LMMS/lmms/pull/5712

also, the detuning is done in a separate window,

I think this could also be fixed via enhancements to our overall automation workflow. The UX required here should be similar to that for automation adjustment directly in the song editor (limited vertical space, etc). See #5541

I'm right now thinking about adding support for stuff like [...] vibrato

IMHO this is yet another improvement that should go directly in the automation editor: the ability to interpolate a sine/triangle/saw/square wave between two points, with adjustable frequency and amplitude.

Other

First of all by default this automation clip is empty, I feel like it should by default have two points that delimit the length of the note,

+1, this should be relatively easy to fix.

@SeleDreams I believe this discussion have been focusing too much on technicalities, lets zoom out and look at the broader picture for a while:

You want to build UTAU support for lmms. And for that you have to be able to automate pitch bend, portamento, vibrato etc etc.
Writing automation concurrently with notes in lmms is a pain. You have to use song editor+piano roll+automation editor all at the same time and there is no way to sync all of these.
But this is not all true. There is the note detuning feature of piano roll and it is a bit simpler to use, still not great.
So to improve workflow for UTAU you want to add more automations similar to the way of note detune, but for stuff like vibrato etc.

Here comes the problem: in lmms we can automate everything, so out of 100 things why would we add only vibrato in piano roll? No, it is better to make this generalized. Then all plugins will benefit.

Some issues you bring up about the note detuning feature comes from the fact that it is not made primarily for your usecase. Vocal synthesis, I would guess, is monophonic most of the time, so in that case, pitch automation could be clip wide. But if you work with chords, note specific automation has a very good usecase.

So the reason you are going after note detune is not that there is a problem with it. It is because you actually want is automation editor integrated with piano roll, and note detune is the closest thing we have at this point.

Now I must conclude by saying I agree with the things said by you and Specular. There is much room for improvement for both note detune and automation in general, and your suggestions are good. Just want you to come at this problem from the right angle.

For me, as someone who was insane enough to made orchestral tracks in lmms, usually with a lot of automation going on, having better visibility of what is automated could be really useful. But how?

semitransparent background in automation editor? This needed of course full synchronisation with pianoroll editor, possibly showing clips from track rendered as long continous piece. I see some possible issues, especially considering how users can be creative with using lmms.
ghost notes in automation editor

LMMS / lmms