Wishlist: a plug-in, script or feature to automatically report key switches on the fly during composition

[ScottChesworth]

I'd like to explore whether there's any potential around getting key switches to be reported automatically, most likely in conjunction with REAPER's template and named notes features. Think of this as a place to collect/discuss thoughts/ideas. I'm definitely not smart enough to implement this, I'd just love to use it. Tagging my man @TimTam at the top, maybe he'll be able to help with the technical.

Scenario:

Key switches are a widely adopted method for controlling articulations when using sample-based instruments. With orchestral libraries in particular, judicious use of key switches is often an essential part of emulating realism. But here's the rub, there's no standardization around where/how key switches are assigned from instrument to instrument, right now, the standards I've observed are that 1) modifying the assignments is typically inaccessible and 2) remembering switches per instrument is incompatible with the small portion of my brain that stays active during composition.

Current workarounds:

1. Some folks take notes. That's a good strategy in terms of pooling resources, but there's no standardization around that either and checking documentation mid creative flow can be disruptive.
2. Real mates share tem-plates. More work on the front end of course, but by using REAPER's named notes feature (usually on dedicated key switch tracks), reporting can be integrated into the project. However, that still involves the user taking their hands off the controller, opening the MIDI editor and scrolling the pitch cursor. For a scatter-brained part-timer prone to imposter syndrome such as myself, anything that interrupts notes being played gives me time to think about what I'm making, which is inherently risky.

Exploration of how auto-reporting could work:

1. I'm not seeing this being a core OSARA feature, mainly because I came up empty when trying to imagine how it would work in conjunction with what we currently provide. Decided to float it here though as this is a good place to reach composers who also have coding/scripting chops. If I've missed an idea here, let me know.
2. My first thought was a ReaScript that could be fed a note range and use OSARA's OutputMessage function to report note names within that range. This would go hand in hand with the description of templating above. But then, a scripted approach isn't sitting right because I'm not sure how that would work for multiple instruments/libraries within a project, plus preservation of imported scripts around key map updates is a bit shakey. Seems like there'd be very real chances for less technically inclined composers to get caught short.
3. A js plug-in reporting through OutputMessage instead, maybe? My thought is that this could have a minimal accessible GUI for configuration and be incorporated into shared track templates wherever relevant, thus tailoring the functionality to each instrument/library. This could be provided both as a standalone download and through ReaPack, our snapshots page and Toni's ReaPack repo would be good spots.

Who can do what?

I'd test the heck out of anything in this area, would help with updating existing templates, could potentially sponsor the work, and would gladly put time into making a free training resource available. Can also contribute childlike enthusiasm during ungodly hours of the night (if that doesn't get you gunning for a finish line, nothing would). :)

Any and all thoughts/ideas appreciated. Let's bat it around in the comments, I'll update this top post if something starts to take shape.

[jcsteh]

The biggest technical challenge I see here is that as far as I know, JS plugins can't call REAPER API functions. ReaScript or C++ code can, of course, but those can't intercept MIDI input in realtime. That leaves writing a C++ VSTI which also calls the REAPER API, which I think is how ReaLearn is implemented. But writing a VSTI is significantly harder than writing a JS plugin, though probably not as hard for MIDI as it is for audio.

[ScottChesworth]

as far as I know, JS plugins can't call REAPER API functions.

@Justinfrankel, can you either confirm or deny this? If it is possible to do, a link to docs would be a huge help.

[ptorpey]

Does this help?:

https://forum.cockos.com/showthread.php?t=212174

This is on the Reaper forum under the title:

js_ReaScriptAPI extension - Cockos Incorporated Forums

Quoting from the post:

“I have uploaded a new extension that may be of interest to other scripters: "js_ReaScriptAPI".

It can be installed via ReaPack (if ReaTeam/Extensions is not listed among your default repositories, you can add the repository manually with https://github.com/ReaTeam/Extensions/raw/master/index.xml this URL), or via https://github.com/juliansader/ReaExtensions/tree/master/js_ReaScriptAPI/ direct download (copy the file to REAPER's /UserPlugins directory).

The idea behind this extension is to make all the useful and powerful functions that are available to C++ extensions available to ReaScripts too. (The resulting ReaScript API is therefore not the result of my own amazing programming skills, but is mostly just an interface to existing C++ functions.)”

--Pete

From: ScottChesworth @.> Sent: Friday, December 30, 2022 3:04 PM To: jcsteh/osara @.> Cc: Subscribed @.***> Subject: Re: [jcsteh/osara] Wishlist: a plug-in, script or feature to automatically report key switches on the fly during composition (Issue #826)

as far as I know, JS plugins can't call REAPER API functions.

@justinfrankel https://github.com/justinfrankel , can you either confirm or deny this? If it is possible to do, a link to docs would be a huge help.

— Reply to this email directly, view it on GitHub https://github.com/jcsteh/osara/issues/826#issuecomment-1368104026 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ADEPTJKMX5X6FMBYJEZEV73WP5L43ANCNFSM6AAAAAATMUX54I . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[ScottChesworth]

Does this help?

Somehow forgot that existed! Speculatively tagging @mespotine, Ultraschall makes use of that extension if I'm remembering right, maybe she'll have ideas/advice.

[mespotine]

I've been summoned? ;)

JS-extension is not related to JSFX! It's confusing because of the name, as the author is Julian Sader whose initials are JS. JS-extension is ReaScript only.

JSFX can not use Reaper's API. I had asked Justin once at askjf.com for how possible it would be to add some kind of extension support to JSFX and he wrote that it would be possible but make JSFX more complex so he prefers it to the way it currently is(hope I quoted him correctly).

One question for clarification: what exactly do you mean with key-switches? It's not shortcuts, I assume?

[mespotine]

In regards of Midi-input, there's MIDI_GetRecentInputEvent in Reaper's Api, that allows getting the midi-input read by Reaper. It had been added for retrospective recording of notes, so you can add played notes to your script that had been played before record start.

It's basically a queue for Midi-input events, that you can iterate over. I have not much experience with it, but the user mpl on the Reaper forum uses it.

[Timtam]

Guys, we don't need to reinvent the wheel here. There already is a REAPER plug-in called Reaticulate by @jtackaberry which does way more than just keyswitches. Its mostly a plug-in which adds a GUI to quickly change articulations through a unified interface which allows more than just keyswitches, but also program changes and those kind of things. It allows to write banks per instrument which control how the various articulations are loaded and keeps track of which one is currently loaded. So that functionality already exists, the only thing we'd need here is accessibility I guess, but I've never looked at it in person and don't know how accessible all the functions already are or aren't, and what needs to be done in order to get this into a more accessible state. We might be able to give some advise / test it though, and can probably work something out to add additional accessible either directly with Jason or within OSARA.

[ScottChesworth]

I've been summoned? ;)

Strange thing, I was just standing in front of a mirror chanting "documentation, documentation, documentation" and BAM, you're here. :)

One question for clarification: what exactly do you mean with key-switches?

In a lot of instrument libraries, developers provide the ability to control the articulation by pressing notes/keys on a MIDI controller keyboard. They're usually notes that are outside of the playable range of the instrument, and dropping them into the MIDI arrangement is a cool way to make each part more expressive. To take some random examples, you might hit a note somewhere right down at the bottom end of the keyboard to control how a bowe was being held when a violin was being sampled, or with a different library, you might hit a note that shifts the position guitar chords were played in during the choruses of a song to get some extra contrast between lines that are panned left and right. These notes that control deeper functionality of each instrument are what we call keyswitches. Visually, it's typical for the available choices to be laid out near the bottom of the GUI in most libraries where they're supported. There's no such standardization around which notes these functions get assigned to when you're triggering via a MIDI controller keyboard though. Essentially I'm hopeless at remembering which notes do what with each instrument, re-figuring that out whenever I haven't used a library for a few days slows me down when I'm trying to get music written, so I'm scrabbling around looking for a way to get speech feedback whenever I hit a keyswitch that's assigned. REAPER already provides a way to name specific MIDI notes, and in theory that could be a useful way to customize the speech feedback we'd like to hear for each keyswitch. Getting those note names we've chosen reported to the screen reader on the fly is gonna be the tricky part though.

[ScottChesworth]

There already is a REAPER plug-in called Reaticulate by @jtackaberry which does way more than just keyswitches.

Ooh la la, this is powerful stuff!

I've never looked at it in person and don't know how accessible all the functions already are or aren't, and what needs to be done in order to get this into a more accessible state.

Nevertheless it's a great suggestion, thanks man. I'll take a poke around and see what I can find out, hopefully sometime tomorrow or Sunday.

[jcsteh]

In regards of Midi-input, there's MIDI_GetRecentInputEvent in Reaper's Api, that allows getting the midi-input read by Reaper.

Thanks so much for this tip. I hadn't come across this before. I implemented a prototype using this in #829.