Add UI for creating and modifying banks

[jtackaberry]

Once the requirements settle during the alpha preview, a GUI is needed so users aren't required to edit reabank files.

[xshill]

Hello @jtackaberry !

I actually started working on a UI to create banks on my fork, and then saw that there was already an open issue for this. I'm pretty motivated to work on this in the next couple of weeks, especially since I'm planning on using Reaper a lot in the coming months and this feature would really improve my workflow.

I was wondering if you'd accept the help on this issue. Also, I'm curious if you had any UI / UX design you had settled on for the project, or if it was pretty open.

It's still a WIP, but the UI I made currently looks like this (but I'm more than willing to change it if you have any comments):

"Create New Bank" button in the banklist:

Bank creation screen:

Icon picker (this screen opens up when you click on the icon button for an articulation):

There's still some things missing:

• Fields for the bank metadata (group, name, etc.)
• Bars to the left of each articulation so you can reorder them, like the banks in the bank list
• Actually creating the bank from the articulation list when you click on "Create Bank" (it just creates a dummy bank right now)
• Maybe a search bar at the top of the icon picker, so you can search icons by name? I'm finding it a bit difficult to find the icon I'm looking for sometimes, since there are so many of them

I think I could split this between at least 2 PRs to make it easier to review: the first one to create banks via the UI, the second one to modify existing banks.

Let me know what you think! I don't want to step on any toes here, so if you have any feedback on how the feature should be implemented I'd love to hear it.

[jtackaberry]

Hi @xshill !

I was wondering if you'd accept the help on this issue.

At this point, yes, compared to where I expected to be with this work item, I think I have to admit that I could really use some help with it.

Also, I'm curious if you had any UI / UX design you had settled on for the project, or if it was pretty open.

In fact most of what I actually have is ideas along these lines. Almost 3 years back I did up a working sketch of what I wanted the bank editor UI to look like:

It's garbage code never intended to see the light of day (and doesn't even work anymore due to changes in Reaticulate and rtk), but it captures the idea. Specifically that:

• The bank editor would be a separate window (triggered via a separate REAPER action, as is necessary for multiple windows)
  • Mainly this is an admission that a fully featured editor really needs more space than the (usually) narrow docked window of the main UI would permit
• Would allow drag-and-drop ordering of the articulation list
• Allow browsing and selecting from existing banks

Not reflected in the mockup, but other goals and miscellaneous thoughts I have accumulated over the years:

• I am not married to that 3 column layout. I could envision the bank list/selector being a ComboBox instead (see below), or perhaps a collapsible left pane
  • I have been warming to (and some users have advocated for) more of a grid/table-based approach for the articulation list, in order to make the most common tasks quick
  • As a general design goal, common things should be quick/easy, and less common things should be possible
  • I am thinking there's much we could borrow from Studio One in this regard
  • In particular I really like the idea of having a GUI based output event list (what S1 calls "activation sequence") in the right-most panel that is reflected in some terse form in the main grid view (under the Activation Sequence column), but which also allows editing the terse form in the grid view directly. Power users can bang out most of what's needed for articulations right from the grid view, while new/intermediate users can use the GUI. And because they are linked, the format/language becomes discoverable.
• It'd allow both creation of new banks as well as editing existing ones
• It would be possible to import banks from file/clipboard as is possible in the main window
• It would support MIDI learn by communicating with the JSFX on the selected (and record armed) track
• It would exist in app/editor, and at some point in the future I would refactor the shared code into lib/ and move Reaticulate's existing GUI into app/main or some such. (I also have future ideas to support a lightweight articulation picker popup window for quick, one-shot access from the MIDI editor or arrange view but this requires some architectural separation of some of the background tasks handled by the main window (like the MIDI feedback/control surface functionality) into a headless "process". I've more ideas than time, unfortunately. :))
• The bank editor and the main GUI would communicate with each other using the same mechanism that the current actions communicate with the Reaticulate window
• All inputs are validated. It must be impossible for users to create syntactically invalid banks using the editor.
• It should be possible for articulations to be activated from within the editor to enable users to validate that the articulation works as specified before committing/saving the bank. Ideally this is even done in realtime as the user modifies the output events.
• It should allow articulations to be copied and pasted within and between banks, because we tend to encounter the same patterns over and over, especially within the same library

One of the key things missing from rtk to support this is a good, flexible ComboBox/picklist. There's rtk.OptionMenu but this isn't a combo box, and even as a pick-list it isn't flexible enough. Some time ago I began working on a ComboBox that supported arbitrary elements with list (such as images, or in general anything subclassed from rtk.Widget), with realtime filtering, nested hierarchies, etc. I imagined using this in many places, such as icon selection, color selection, and naturally within Reaticulate's main GUI using it for selecting banks (which is really obnoxious right now with the OS-native menu). But then life got busy and I never finished it, and seeing it as a prerequisite for the bank editor, everything ground to a halt. And so it goes.

I think my basic problem is that I'm still working on accepting that perfect is the enemy of good. Or at least good enough. :)

So if you'd be willing to tackle at least part of this vision and get something basically usable, even if it's not as fleshed out as the ideas above, that'd be fantastic. I'd be happy to review PRs or WIP in your fork. (Where appropriate, please follow this style guide).

I do think what you have so far, just from the screenshots you shared, is a great first attempt, especially considering the constraints you started with (i.e. building it into the main GUI which needs to be usable with quite narrow widths and rtk's own limitations). Splitting it out into a separate window (via separate action) will give a lot more flexibility. And I think we could do a lot worse than lifting from the S1 video I linked above.

Then perhaps we can divide and conquer some of the work, where I could get rtk.ComboBox into some working order and one of us could plug that into the editor when the time is right.

Hopefully I didn't scare you off. :)

[xshill]

Thanks for the (very quick) reply!

The mockup looks great! I'm down with moving the editor to a separate window. Like you said I think it would make the UI much more flexible if we're not constrained by the size of the main reaticulate window.

The Studio One editor does have some great ideas. I don't think there's a table widget right now in rtk (please correct me if I'm wrong, I didn't see it in the documentation though). But we could keep the articulation list in the middle, and open the selected articulation's details in the right panel. At the top of the right panel, you'd have the articulation details (name, icon, program number, and so on). At the bottom, you could have a scrollable list of outputs.

Although the bank selector looks great, we don't necessarily have to show that all the time. I'd personally (at least for a v1) keep the bank selector in an OptionMenu. That gives us space to show the bank details on the left, so we can have the articulation details / output list on the right.

To edit articulation icons, I think we could show a "reactive" popup window that instantly shows the changes you're making in the main editor window.

All inputs are validated. It must be impossible for users to create syntactically invalid banks using the editor.

I think this is important, although I'm a bit turned off by the complexity of adding a reactive validation mechanism. Are you ok with just validating before we save the bank as a start, as opposed to validating in real-time?

The bank editor and the main GUI would communicate with each other using the same mechanism that the current actions communicate with the Reaticulate window

I haven't really touched on that on my work yet, what is there to communicate between both windows?

And one more question: what is the inherit button in the bank details? Is it so that the new bank we're creating will have all the articulations of this bank we're inheriting, plus whatever articulations we're defining?

For a v1, I think I could make something like this:

Which would have the following features:

• Both create and edit banks
• The bank optionmenu is basically a load menu, it also has an option to create a new bank
  • The default is to create a new bank, but the option in the menu would allow you to go back to a fresh new bank if you had previously selected an existing one
• Articulation and output lists can be reordered
• When saving a bank, validate every single field in the UI to make sure there are no errors
  • If possible, report all errors at the same time so you don't have to fix the first error, save, fix the second error, save, fix the third error...
• When saving an existing bank, we can ask the user to confirm that they want to overwrite the bank
• The test button can be used to test the articulation settings.

WDYT?

[jtackaberry]

I don't think there's a table widget right now in rtk (please correct me if I'm wrong, I didn't see it in the documentation though).

Not as such. rtk remains pretty anemic on widgets.

But rtk's foundations are pretty solid, and the building blocks are all there to create something that looks and feels remarkably like a proper table: notably, boxes and the expand and minw cell attributes.

So I've sketched something out:

The expand cell attribute is doing a lot of heavy lifting here. And we can bias the column sizes by using different expand values; for example, the Output column gets most of the available space, since that's where the longest strings are likely to be. I could have included maxw cell attributes to the Program and Color columns as well, since there's no point in these getting significantly larger than their minimums -- I just didn't think of it until typing this.

(The screenshot shows tab/shift-tab focusing which is in my dev branch of rtk but not committed yet.)

The code won't win any awards but it should get the point across. Here articons and reabank are assumed to be require()d elsewhere, as in the existing Reaticulate code base.

local function main()
    local window = rtk.Window{w=800, h=600, padding=10}
    window:open{align='center'}
    -- Cell attributes for each column in the table
    local cellattrs = {
        {expand=0, minw=32, halign='center', rpadding=15},
        {expand=1, minw=100},
        {expand=0.5, minw=75, maxw=100},
        {expand=1, minw=100, maxw=120},
        {expand=3},
    }
    -- Articulation list
    local data = {
        {'legato', 'Legato', '20', 'legato', 'cc:32,20'},
        {'note-whole', 'Long', '1', 'long', 'cc:32,1'},
        {'spiccato', 'Spiccato', '42', 'short', 'cc:32,42'},
        {'staccato', 'Staccato', '40', 'short', 'cc:32,40'},
        {'pizz', 'Pizz', '56', 'short-light', 'cc:32,56'},
    }
    -- Hack. Proper implementation would use user defined colors first, only falling back
    -- to defaults if not defined.
    local artcolors = reabank.default_colors

    local tab = window:add(rtk.VBox{minw=450})
    -- Heading row
    tab:add(rtk.HBox{
        padding=7, bborder="1px #404042",
        rtk.Text{"Icon", color="#aaaaaa", cell=cellattrs[1]},
        rtk.Text{"Articulation", color="#aaaaaa", cell=cellattrs[2]},
        rtk.Text{"Program", color="#aaaaaa", cell=cellattrs[3]},
        rtk.Text{"Color", color="#aaaaaa", cell=cellattrs[4]},
        rtk.Text{"Output", color="#aaaaaa", cell=cellattrs[5]},
    })
    -- Add all rows with the same cell attributes
    for _, row in ipairs(data) do
        local icon = articons.get(row[1])
        tab:add(rtk.HBox{
            padding=4, bborder="1px #343437", valign='center',
            rtk.Button{
                ref='icon', icon=icon, padding=1, color=artcolors[row[4]], cell=cellattrs[1],
                onclick=function()
                    log.info('TODO: open icon picker')
                end,
            },
            rtk.Entry{row[2], padding=2, w=0.8, bg='transparent', cell=cellattrs[2]},
            rtk.Entry{row[3], padding=2, w=0.8, bg='transparent', cell=cellattrs[3]},
            rtk.Entry{
                row[4], padding=2, w=0.8, bg='transparent', cell=cellattrs[4],
                onchange=function(self)
                    self.refs.icon:attr('color', artcolors[self.value] or artcolors.default)
                end,
            },
            rtk.Entry{row[5], padding=2, w=0.8, bg='transparent', cell=cellattrs[5]},
        })
    end
end
rtk.call(main)

Lots more UX finesse needed here, but as you can see with enough elbow grease you can accomplish quite a lot with the existing rtk constructs.

To be clear, I'm not asking you to do this initially. I won't complain if you feel sufficiently motivated of course, but your v1 proposal sounds good to me and I don't want to drag you down into scope creep hell like I've done to myself over the past ... well, I see this issue has been opened for 6 years now. 🤦‍♂️

Mainly I just wanted to demonstrate how much you can do from first principles. If you have web development experience, think of it as hand-rolling components with HTML, CSS, and Javascript.

Although the bank selector looks great, we don't necessarily have to show that all the time. I'd personally (at least for a v1) keep the bank selector in an OptionMenu. That gives us space to show the bank details on the left, so we can have the articulation details / output list on the right.

I'm with you. Your sketch is excellent, and makes the point nicely. I really like your idea: there is an intuitive flow to these three panes, bank-level attributes -> articulation list -> selected articulation details.

I think this is important, although I'm a bit turned off by the complexity of adding a reactive validation mechanism. Are you ok with just validating before we save the bank as a start, as opposed to validating in real-time?

Yep. Inline (I wouldn't call it reactive in this context, which has a different connotation) validation can very easily come later.

I haven't really touched on that on my work yet, what is there to communicate between both windows?

These flows spring to mind:

• Bank Editor -> Main window
  • When changes are saved in the bank editor, the main window needs to be signaled to reload the reabank file so the changes are immediately reflected in the GUI
  • MIDI learn should go through the main window -- given how communication with the JSFX works (a pseudo-IPC protocol leveraging REAPER gmem buffers), if two ReaScripts are doing this concurrently, the data is sure to get corrupted. Unless you can think of a different way to accomplish MIDI learn without enlisting the JSFX's help.
• Main window -> Bank Editor
  • If the bank editor window is already open and the user clicks "Edit bank" or "Create new bank" button from the main window, the active bank editor needs to be signalled to change its state, if necessary, prompting whether existing changes should be first saved

Between BaseApp:send_command() and BaseApp:check_commands() (intended to be overriden by subclasses), the BaseApp class should have everything needed to handle this cross-window communication. Both the main window and the editor window would subclass BaseApp.

Here's an example of send_command() -- and oh, would you look at that, somehow some of my very early prototyping of the bank editor slipped into the main app code base. :) And here's the main app's implementation of handle_command().

It's pretty crude, really only useful for passing one or more simple scalar values, but hopefully should be sufficient.

And one more question: what is the inherit button in the bank details? Is it so that the new bank we're creating will have all the articulations of this bank we're inheriting, plus whatever articulations we're defining?

Yes, it's the clone attribute defined at https://reaticulate.com/reabank/#attributes-for-bank-lines

clone needs to be updated to support bank GUIDs, and that will be the proper way of referencing cloned banks going forward.

I called it "Inherits" in the GUI because that makes more logical sense. The bank editor has an opportunity to fix/improve some of the terminology to be more user-approachable. This could even include the term "bank" itself, which makes sense from an implementation perspective, but "articulation map" is the more canonical term.

In any case, whatever the terminology, the editor should eventually make liberal use of tooltips to help discoverability.

Which would have the following features [...] WDYT?

Sounds great. I think at this point just getting something out that we can iterate on is key.

Thanks!