API for User Scripting With Lua

[eliot-akira]

This is a high level bucket list for a minimal scripting engine. Some of these may change or be removed.

• [x] Lua Node - Internal plugin which runs a Lua script for audio/midi processing
• [x] Integrate sol3, update to c++17
• [x] Finalize Lua Node script api
• [x] define types of scripts to run
• [x] Lua Proc <-> UI communication. RingBuffer to pass parameter values?
• [x] User script registration
• [x] save/edit scripts directly to session
• [x] and much much more!

• [x] Bind as much of the api as possible

  More issues can be found in the Scripting project.

Original Issue Below: Would you consider adding some kind of API for user scripting?

A couple of example use cases: a MIDI effect that applies octave shift or algorithms like arpeggiation; a MIDI input/source as a programmable sequencer.

As for examples in the wild.. In Ableton, users can provide MIDI "remote scripts" in Python. In Bitwig Studio, a similar feature exists with JavaScript (examples: https://midiscripts.net/). With Max for Live, apparently it's possible to have a "device" that communicates with external Node.js processes (max4node).

Lua seems to be popular as a light-weight embedded language suitable for such purposes. On their home page, they describe it being used in Photoshop, games, etc. By chance, I recently came across a Scheme-based language called s7, used in Radium music editor and Common Music.

To explore possible options, I looked at how big Lua is - 225Kb. I know of a very well-written JavaScript implementation called QuickJS, it's 190 Kb. (The latter is used in the design application Figma, for user-written plugins - reference.) Then I looked at how big Element.app is - wow, is it correct that it's only 96Kb?! Amazing..

An alternative to an embedded language might be, like max4node, to allow a "device" in Element to communicate with external process via UDP socket. That way, the client could be written in any language.

Well, just a feature idea - I think it would unlock lots of possibilities, like user-land plugins that could be programmed in a simpler/safer language than C.

[mfisher31]

Absolutely! This was already in the plan, but not moved to GitHub yet. Lua looks like a good option, JS could be cool too. JUCE has a built in parser that'd integrate nicely with the model layer (items in the session dir).

Have to be careful that scripting libraries don't force the usage of static data though, otherwise can't be used in the plugin versions. For example I looked at pybind11 C++ python bindings, but they require static registration of objects which won't work in the plugins.

[mfisher31]

Oh and thanks for all the links. I'll definitely be looking at those. The thing that worries me when using any scripted language for realtime processing is... audio drop outs :(

[mfisher31]

You know for MIDI processing, scripted languages would work pretty good. Would have to segregate MIDI event processing from the Audio thread (also in the long term plan). One way or the other Element WILL get scripting :)

[eliot-akira]

Wonderful! I'm glad to hear you're open to the idea and were already thinking of adding scripting capability.

Running user scripts in a separate thread makes sense, to avoid affecting audio performance. In any case I imagine audio processing might be too much to ask for an interpreted scripting language, and that MIDI is more suitable, as you say.

I really like the idea of sharable scripts, like this community-contributed library of Max devices. I took a look at one of the devices, it's basically in JSON format. Just now, I checked Element's .elg and .els, I see they're in XML. Well, I can totally see this kind of data/structure, or like stream of MIDI events, could be manipulated via user script.

Anyway, I'm looking forward to seeing Element grow. Thanks!

[eliot-akira]

Just to add a note: I found a C++ wrapper for QuickJS, which looks like it would simplify using it as an embedded language, and can be included as a git submodule.

Looking at the source, I suspect it's several times larger than Element itself. It might be more ideal if user scripting could be an Element "plugin" of some kind, external to the core app. I think I'll play around with the idea on a dev branch, and try to create a Node class with a simple textarea, based on some of the MIDI-based Nodes in engine/nodes.

[mfisher31]

Sounds good. Let me know if I can help in anyway. Right now there's two main Node types. 1) those that are based on juce::AudioProcessor and 2) those that are not. the no. 2 type would be most appropriate for this kind of thing.

Just noticed you said midi-based nodes. Most of these are of the type 2 variety.

UIs for new nodes should go in gui/nodes or hidden in the .cpp file. Sorry for so many rules :)

[mfisher31]

oh, and without looking first... engine/InternalFormat.cpp <-- nodes created here NodeEditorContentView.cpp <-- node guis created here for the "Node" panel WindowManager.cpp <-- node guis created here for plugin windows.

Still need a "content manager" of sorts to DRY up UI factory functions

[eliot-akira]

Thanks for the pointers! This will be a nice hobby project for me - will let you know if/when I make any progress. :)

[mfisher31]

No problem. Oh, and loadable modules is kind of already working. Something to also consider is writing these things as LV2 plugins (yes LV2 will work on Mac/PC)

[eliot-akira]

Baby steps.. I managed to:

• Create an empty MidiScriptNode and MidiScriptEditor by referring to MidiProgramMap
• Add the node to the app
  • Define new ID in engine/nodes/BaseProcessor
  • Pass node/plugin description in engine/InternalFormats
  • Instantiate node in session/PluginManager
  • Instantiate editor in gui/WindowManager

Cool! Now I have a blank node for exploring what messages are going through. First I'll figure out how to display MIDI notes as text. I found that JUCE has a CodeEditorComponent, so I'll see how that works next.

For the interpreter, I attempted to bring in QuickJS but had difficulties getting it to build, due to my lack of knowledge about compiler options, linking, etc.

Then I tried Lua. Woo, it was so easy, a beginner like me could embed it. The source files are friendly so I can see how it integrates with C. I like the aesthetics of the language too.

Anyway, lots of fun learning how Element is built, great stuff in JUCE, basics of C++ itself..

---

I found x42, who has tons of LV2 plugins for further study.

Oh, by the way, in the Plugin Manager, I don't see any LV2 plugins, or an option to scan for new ones. I've got a few in the right location (/Library/Audio/Plug-Ins/LV2 on macOS). I've also got a bunch of them in /usr/local/lib when I brew installed lv2.

[mfisher31]

Just a sec on that. I"m experiencing the same thing. A change somewhere stopped LV2 from showing up there. Are you on discord @eliot-akira ?

[eliot-akira]

Another approach I'd like to explore in parallel: a SocketNode that opens a UDP socket to communicate with external apps, with just a single field in the UI to set the port number. I suppose that's much simpler/smaller scope, and keep the focus on the API/message protocol that Element would provide. I saw your note about Open Sound Control - oh, JUCE has an osc module. OK, I'll learn more about it.

[mfisher31]

Ok, I pushed some changes to master which re-enable LV2 in the plugin manager. To fix AudioUnit scanning, I had to update JUCE along with libs/jlv2 and libs/kv. To if you merge/checkout master you'll need to git submodule update

SocketNode sounds like a cool idea. You may look at implementing a new AppController::Child if it needs to interact with higher level app functions. Having a single SocketNode could also do cool things if it were to generate MIDI messages and/or LV2_Atom messages. The former is yet-to-be added as a port type on nodes (coming soon, I hope).

BTW, x42, is an LV2 genius. Tons of cool stuff over there!

[mfisher31]

Oh and https://docs.juce.com/master/classDatagramSocket.html, a low level UDP impl.

[mfisher31]

@eliot-akira I'm thinking we try to get a minimal scripting engine added for 0.43.0 !

[eliot-akira]

@mfisher31 I've been following your progress on the feature/lua branch, nice to see how it's getting fleshed out! I think once you've laid down the foundation and basic pattern for adding bindings, I could contribute and try filling out more of the details in LuaBindings.cpp.

I was looking at how a very small LV2 plugin is written, like this one: https://github.com/dsheeler/tremelo.lv2/blob/master/tremelo.c

From how Element's Lua script API is shaping up, it seems it's not too far from being able to achieve something like that - mainly, defining a run or process function that takes an audio/MIDI buffer and applying transformations on it.

I've been reading through Ardour's manual section on Lua scripting, and how they have different types of scripts (Editor Action Scripts, Session Scripts, DSP, ..). The DSP examples are very similar in structure to the LV2 plugin. Here's their example folder with many Lua scripts, they look useful as reference.

[mfisher31]

By all means... bind away! Even though the C++ code follows the JUCE style, I think snake case is more appropriate for Lua. I'm sure you saw that in the bindings I've already done. BTW, there's a tests/LuaTest.cpp started if you get sick of running the GUI all the time to try things out. Individual tests can be run by doing:

build/bin/test-element "Category" "slug"

.. or create new ones. I use the test suite all the time to try new things.

Indeed Ardour is full of good examples! Great minds think alike as I've been doing the exact same thing!

My next challenge is to get parameters working. My initial thought was to recycle JUCE's AudioProcessorParameter, but decided against it for a few reasons:

1. They are tied to AudioProcessor directly
2. They can't be used for an "output" parameter
3. Can't use them for CV ports (coming sooner or later)

[eliot-akira]

Thanks for the tip on using tests for faster dev cycle, makes sense!

..Hmm, I ran ./waf build --test and build/bin/test-element, and I don't see LuaTest in the tests that ran. I checked the waf script and unit test convention used, but couldn't find why it wasn't included. I also tried setting env variable EL_USE_LUA.

---

About AudioProcessorParameter, that does sound like a higher abstraction is needed to wrap raw JUCE class/methods to be used by Lua scripts. I see that these will become the basic building blocks for user-written plugins, exciting idea!

[mfisher31]

LuaTetst should be in master as well as feature/lua: build/bin/test-element Lua node <-- run just that test.

[mfisher31]

Oh, I see what may be the problem --test needs to go along with configure

./waf configure --debug --test
./waf build

[eliot-akira]

Aah, yes, I ran the above and see the tests got rebuilt. Cool, looking forward to exploring this!

[steveschow]

Just want to add a couple comments about scripting, glad to hear something is happening in that regard with Element.

I have been using LogicPro's Scripter (javascript) for a few years now and I have become quite adept with it. Its very useful, within LogicPro. It only handles midi, no audio.

There are third party scripters on the market already, one free one called ProtoPlug, uses Lua. Another commercial one from BlueCatAudio uses angelscript, which I don't love actually, but there are technical advantages. both of those can render audio in addition to processing midi. ProtoPlug is based on JUCE too by the way, might be worth a look if you are leaning towards Lua.

One advantage of LogicPro's built in scripter is that it gives access to a few things that are not possible for normal AU plugins. For example, Scripter can detect articulationID in midi events, which is not exposed through the normal AU api.... So anyway the point I want to make is that what would make a scripter extra interesting in Element is if it exposes Element-specific features in some way that these third party AU/VST scripter plugins would not be able to.

I also want to point out that the scripting API for both ProtoPlug and BlueCatAudio is actually very simplistic and could be better. Its literally easier to build midi filters in JUCE then it is in either of them, because they basically just give you the raw buffers and you have to do everything yourself, whereas in JUCE there are objects to keep track of the midi queue. So again, what I'm saying is that scripting will be MUCH easier for people if you provide some higher level script functions.

LogicPro's Scripter has a much better developed API in terms of higher level midi functions, but I think they could have even gone further then they did. But at least its done in a way that very simple scripts can easily be written to handle a lot of simple midi tasks.

It also provides access to LogicPro's automation system and has simple GUI capabilities.

I could easily see situations in Element, for example, where you might want to use scripting to dynamically switch signal routing, or things like that. Think PlogueBidule, but being able to use a scripting language instead of arcane and complicated electronics logic like is done in PlogueBidule.

[eliot-akira]

Thanks for your input, @steveschow - interesting information! I gathered some links below for reference.

• Protolog
  • GitHub repo
  • Website
  • Lua API reference
• Logic Pro Scripter
  • Logic Pro X manual: Scripter plugin (PDF)
  • Feels So Good on LinnStrument with Logic Pro Scripter (In depth about API and example usage)

The Protolog source code looks useful. It provides a Lua interface on top of JUCE, similar to how Element has started with Lua integration. Looks they got quite far with it, before development stopped a couple years ago, it seems. There are probably some ideas in there that Element could incorporate.

I haven't had much time to explore this direction (hi @mfisher31 !) - but I hope to come back around to the fun. I recently got myself a Raspberry Pi with an audio interface, and I plan to run Linux with Element (standalone, maybe GUI/headless). Ability to write and run Lua scipts would be wonderful.

[steveschow]

Its called "ProtoPlug", but yea, looks good.

Yea he did some pretty cool stuff with Lua and ProtoPlug. I myself prefer JavaScript. JS is built into JUCE, but my understanding is that its a very basic javascript engine. It can be extended of course, but I am not sure how it would perform. would probably be better to use a better JIT compiler such as google's. But then that gets more complicated. Easiest first go would be use JUCE javascript and see what can be done with that.

The key to making things perform well is, as already suggestion, keep the javascript or lua execution out of the real time thread for one thing I guess, but also managing critical objects such as event objects, in C++ with bindings to those objects in Lua/JS, so that you don't have to constantly create and destroy a lot of event objects neither in the scripting language nor behind the scenes with bindings.

But anyway, my experience with non-programmer musicians that want to script with LogicPro Scripter, is that their ability to understand OOP and even pretty simple javascript is very limited. In order to make something like this usable, need to have a high level abstraction that makes it dead simple to do common tasks. I had many ideas for creating a framework to use with LogicPro Scripter, but the problem is that it does not provide any ability to include or require anything...so you always have to copy and paste all your shared code into every script, and pretty much becomes very unwieldy at that point. Really Apple needed to provide a better framework to begin with. But what they did provide is still a little easier and higher level then both ProtoPlug and BlueCatAudio Plugn'Script.

Also these scripters primarily deal with just filtering or generating audio or midi...they function like a normal plugin, but I think in element there is a huge potential for being able to create script-controlled graphs and stuff like that way beyond simply filtering the audio/midi signal.

[steveschow]

Imagine for example... OSC commands...hitting a script....and doing all manner of control over Graph routing, setting parameters in plugins, etc.. in who knows whatever ways.. Then people can setup their iPad OSC controller and control a graph in detail with the ability to script decision trees, hold data structures of information, possibly read JSON from text files to configure things, etc.. scripting opens a lot of possibilities.

[eliot-akira]

Yes, I feel it too, the possibilities are great for Element scripts.

In an earlier discussion I mentioned QuickJS. It's an excellent light-weight implementation of JavaScript, designed to be embeddable like Lua. I've got more familiar with it since then, and it provides a nice virtual machine to run scripts, with built-in mechanism to prevent infinite loops or set a timeout. It's definitely superior to the minimal script engine in JUCE.

I kinda agree with you that JavaScript feels friendlier than Lua, only because I'm more fluent in it. On the other hand, Lua is a small and beautiful language - performant, popular with games (I heard) and in academia, developed and used around the world over decades - so I think it's plenty suitable. It's a quirky and cute language, I like it.

Oh, ProtoPlug, not Protolog. Haha, I guess my brain approximated Prolog. I plan to read through its code base, I'm sure I can learn from it.

keep the javascript or lua execution out of the real time thread for one thing I guess, but also managing critical objects such as event objects, in C++ with bindings to those objects in Lua/JS, so that you don't have to constantly create and destroy a lot of event objects neither in the scripting language nor behind the scenes with bindings.

For this, I know @mfisher31 considers every detail for cleanliness in terms of memory and performance. Element is quite a work of engineering as well as art. So I would trust his judgement on these aspects of integrating a scripting language.

In order to make something like this usable, need to have a high level abstraction that makes it dead simple to do common tasks

I'm very hopeful that, given a "toolbox" of primitives, much of this - like a framework - can be built in user land. For that, like you said, the ability to load/save/require modules would be key. Then an ecosystem can grow around it.

---

I was curious to ask - in Logic Scripter, I'm guessing there's a global timer that emits events (start/stop transport, tempo or time signature change, start of beat or measure..) for scripts to listen to? I'm still a bit foggy on how a script can, for example, loop a sequence of MIDI notes based on an array data, in sync with other plugins or scripts, etc. I suppose I need to read deeper in the links I posted. :)

Edit: Ah, right, a processMIDI function with GetTimingInfo.

[steveschow]

There are a couple callback functions... processMIDI is one, there is another one used more often called handleMIDI(event), which passes a midi event object to the script.

processMIDI is called periodically, usually once per process block, with no buffer. So the script can then schedule midi events from there at any time.

handleMIDI is called only for pass each midi event to the script as a callback argument. There is no buffer like in JUCE and other scripting plugins. The musician doesn't have to know anything about looking at a buffer and manipulating it. They just receive a callback for each midi event, and then they have to call an Event.send() method which basically will queue that event (or other events) back to the internal buffer.

There are a couple other callbacks:

Initialize()
Idle()
ParameterChanged()

Let me know if you have any questions about LogicPro Scripter. I know it well enough to write a book by now...

[steveschow]

The GetTimingInfo thing is a little different. By default the script does not have access to sample time of each event object passed into handleMIDI. The assumption by the framework is that any events queued from there will be on the same sample time as the event passed in, hidden to the script. There are some ways to schedule them later like Event.sendAfterMilliseconds and stuff like that.

However, when they use the GetTimingInfo structure, they have to set a certain flag in the script which turns on the sample time property in each event (its actually counted as beat position, rather then sample time). And you can set the same attribute before calling Event.send() to schedule events exactly on a certain beat position (ie, sample time). The GetTimingInfo function returns attributes about the current process block such as meter, tempo, beat position of the start of the process block, beat position of the end of the current process block, etc.

[steveschow]

The way apple did it, for the mass majority of simple midi scripts, its dead simple to script for. And it is, very easy and simple to comprehend. That being said, I have written scripts that are thousands of lines and get complicated fast...

[eliot-akira]

I appreciate the clarification, good food for thought..

This is right up my alley of interest, how to design a friendly programmable interface, especially for music. It's interesting that processMIDI and handleMIDI do not pass a MIDI buffer. For audio processing a buffer makes total sense in the callback, but for MIDI, I was having difficulty picturing how to translate buffers to scheduled events and global tempo/beat.

Right, so the process/handle callback can use Event.send, which accepts beat position to schedule a MIDI event.

..had many ideas for creating a framework to use with LogicPro Scripter.. I know it well enough to write a book by now..

Now I want to pick your brain more. :) I get the feeling that you know about the details of what's needed from the user perspective.

If you were to design a scripting interface for Element based on your knowledge of other similar systems - how would you describe the core concepts? (Say, in JavaScript or even pseudo-code.) I'd love to understand what functions and data would be involved on the script side, to make it easy(er) to use.

[steveschow]

I have had some ideas but they are complicated to explain. I think it would first make more sense to itemize what specifically would be the kinds of scenarios where scripting in element would make sense rather then just making a juce plugin. A lot of musicians aren’t up to c++ so there is always that, but those same users won’t do much with a scripter either.

I think anything involving element’s specific capabilities, in a way that would facilitate more custom experience with element.

But anyway I just think a lot of people need to use a scripter for simple tasks like inserting keyswitch midi events in the fly and things like that. You want those kinds of operations to be dead simple. I think Apple did the best job so far of the scripters thst are out there at keeping it simple. But it doesn’t do audio and has some things that could have been done better. There are still some basic tasks that usually cause typical musicians to give up using it because it requires some tricky scripting for what should be an easy task. Protoplug is even worse that way.

I have been a prominent contributor on a logicpro scripter forum and after seeing it happen over and over the vast majority of people seeking help are not able to write or understand a lot of programming; the fake it through mostly. Certain tasks can only be done with at least a bit more programming prowess and they usually give up.

Apple could have provided just a little more abstraction and api in order to make it dead simple to do a larger number of typical tasks without having to get into programming so much.

When I get to my computer I will try to add some links for further thought. And will expand more about what I think Apple could have done better

[mfisher31]

Hello! Alot of good ideas in here. Definitely want to go as high-level as possible for the scripting. I went with Lua for two main reasons 1) easy language and 2) is stack-based so can execute in a realtime thread.

Fundamental scripting is already implemented as well as a LuaNode which is exactly what it sounds like. I'm at the point now where I'm looking for ideas to get started with when it comes to application/session/graph/node specific bindings.... GUI too!

Oh, and hi @eliot-akira ! Nice to see you're still hanging around the repository. Work on Element has been kind of slow on my end. Really hoping to solidify the scripting engine by the end of summer!

[steveschow]

Fill me in, what is luanode and what kind of fundamental scripting is there now? I will take a look

[steveschow]

When I get to my computer I will expand some thoughts about what logicpro scripter could have done better, and protoplug also

[steveschow]

First I will comment more about LogicPro Scripter, which I am very very much more familiar with. Here is a forum that sometimes has some discussion related to it. The main take away from that forum you can see users that have come asking for help how to script something and see the kinds of tasks they tend to want to do:

https://www.logicprohelp.com/forum/viewforum.php?f=45

Here is one example LogicPro script that I happen to have on gitlab, its not necessarily the most elaborate one I have done, but anyway you can see something for real and perhaps get an idea about how some of the LogicPro Scripter API works. If you have LogicPro, you can try it out. (make sure to read the wiki section too)

https://gitlab.com/dewdman42/midimonitorlpx

Alright so first the good things about LogicPro Scripter:

1. javascript. After playing around with lua a little bit, I much prefer JS, but granted I haven't spent that much time getting more into lua. I think ultimately either language will have pros and cons. I do like that there is a bit more object oriented capability in javascript. Javascript seemed to have a lot more built in functions and classes for stuff like Math, Strings and a few other things, but I probably just didn't get into lua enough to find out more. Note, that even in LogicPro Scripter is not anywhere close to full Javascript as you would have available in a browser. There is no javascript gui stuff, for example. No file system access, etc.. It is very much a sandbox with just enough added beyond basic JS language to support midi processing.

2. Performs well. Apple did whatever they needed to do in terms of binding javascript Event objects to underlying objects inside LogicPro, the RT thread handling, etc.. Its performs very well. I have not really ever had any problem with any noticeable CPU slams by any script I have made. I think it might be possible if you did something like call new over and over in a loop within one process block just to see if you could, but anyway, whatever Apple did under the covers with binding is very well done and it performs extremely well.

3. Simple callbacks. It provides some simple callbacks without a buffer to parse. Instead, Event objects are passed to the user callback script, one callback for each event. So for example, instead of this pseudocode:

function processBuffer(buffer) {

    // loop through buffer looking for midi events and doing stuff, 
    // put back into the buffer whatever you want forwarded on

    while(buffer.isMoreinBuffer()) {
        event = buffer.getNext();
        event.velocity = event.velocity - 10;
        buffer.set(event);
}

Apple took this approach

function HandleMIDI(event) {
    // change the event object and resend it, or add new events if you want
    event.velocity = event.velocity - 10;
    event.send();
}

The Event objects have a class definition and have a hierarchy as well...and yes polymorphism can be used in simple cases to identify what kind of event it is and call the right method, etc..

The above is just a little more simple for typical musician script coder that honestly will become confused by buffer manipulation.

Here are all the callbacks:

HandleMidi(event)
ProcessMIDI()
ParameterChanged()
Idle()
Initialize();

4. Modest GUI. The GUI capabilities of LogicPro Scripter are good enough most of the time and very simple to setup. They cannot do as much as JUCE by a long shot. But probably someone that wants to do a more elaborate GUI should just use JUCE. The approach used for their GUI is very very easy to setup and is automatically tied to plugin parameters. Every parameter is automatable by LogicPro also. its also possible to connect these parameters to other plugins in the same channel strip in LogicPro..but I find that particular thing cumbersome.

   GUI is built by simply declaring an array full of JSON data. For example the following would create a single checkbox on the GUI and associate it as a plugin parameter(saved with the DAW project, etc):

var PluginParameters = [{ name: "Channel Pressure",type: "checkbox",defaultValue: 1}];

5. JSON. Javascript comes with a JSON-like approach to notate and program data structures, which I find extremely useful. Not sure what is possible in Lua. Javascript in general I find to be extremely flexible and capable of a lot of cool tricky ways to track incoming data into structures using JSON and dynamic arrays, etc.. But...as I will explain later, that is also beyond what the typical musician script-coder is going to understand and hence...could be better.

6. OK helper functions. It could use a lot more, but here are some of the included API functions:

Event.send()
Event.sendAfterMilliseconds()
Event.sendAfterBeats()
Event.trace()
Event.toString()
MIDI.noteNumber(name)
MIDI.noteName(number)
GetTimingInfo()
UpdatePluginParameters()
GetParameter()
SetParameter()

7. Internal midi event attributes. Scripter is able to detect articulationID in midi events, which is actually a big deal and one of the main reasons I use it. This only applies to LogicPro, but LogicPro provides an articulationID feature, which is outside the midi spec..its extra data that is passed around in midi events in LogicPro...and can be used to trigger activity in Scripter. Very useful.

Alright so anyway, I could go on forever, but basically I think the above highlights perhaps what is perhaps a bit more developed in LogicPro Scripter compared to other scripters out there like Protoplug. I feel the callback interface without a buffer is more simple for typical musician script-coder and can be a good thing.

Now what could be better about LogicPro Scripter:

1. There is no built in features to store sequences of midi events for whatever purpose. Its always possible to build that with javascript, but again, typical musician script-coder's head will explode. No ability to store a simple sequence and ask for it to be played. You can of course do this manually with enough javascript, but they should have built in functions for this.

2. There is no tracking or management of matching NoteOn to NoteOff. Inexperienced coder can get into problems with hanging notes and sometimes it just makes their head explode trying to do a simple thing like delay a note (and its note off too) by a certain amount, for example. Doing that in LogicPro scripter means tracking the sustained notes in between callbacks. Apple should provide API functions for that kind of thing, and they don't.

3. There is no audio processing

4. There is no ability to detect or send sysex.

5. There is no access to OSC or any other networking

6. There is no file access and no ability to require other libraries. So there is no way to create a user-defined reusable class and reuse it easily across different scripts, or share it with other people. There is no way to read in JSON or XML from a file, etc. Everything has to be contained within the monolithic script itself.

7. GUI is ok, but many people wish it could do more. No tables. Hard to deal with some situations involving more than a dozen or so parameter values.

8. No access to LogicPro's key commands or other aspects of the program. It is simply a midi filter tool. It can be used to control automatable parameters of other plugins within the same channel strip, which is useful, and those can be connected to smart controls also, but other then that, its a sandbox, you can't control anything else in LogicPro. What about markers? Nope. nothing else.. just the basic midi.

9. There is no way with Scripter to report latency to the host, in cases where the script is doing lookahead. The nice thing is you can schedule midi events way out past the current process block, so that's cool, but if you actually wish you could move events earlier in time using lookahead, you can't do it. You could not, for example, write your own quantizer because of that.

10. many more helper functions would be useful, typical tasks like converting beat position to sample time, milliseconds to beats, beats as a fraction vs beats as ticks, chord and scale handling, transposition functions, quantizing functions, humanizing functions, note-tracking( what is sustaining), controller-tracking (keeping track of the current running value of CC's), etc.. There are many things actually.

That's enough for now. I will be happy to field any and all questions in the future about LogicPro Scripter...

[steveschow]

ProtoPlug

I haven't spent that much time with it to be honest. I tried a few simple things at one point and meant to attempt to convert some of my LogicPro javascript to Lua to see what could be done but I more or less decided to start investing my time into JUCE.

ProtoPlug uses the buffer approach and is more like a direct binding around JUCE. But they should have gone further and provided script bindings to the JUCE sequences and other stuff like that. It needs even more high level functions then LogicPro Scripter needs it. It would make sense to continue binding simply around JUCE, but as I said above I think a lot of typical musician script-coders will not like working with buffers and queues and stacks and all that.. They need SIMPLE. I think Apple's approach with a higher level of abstraction is better.

So some of that stuff should be expanded in ProtoPlug but the dev hasn't done any work on it in quite a long time. So I consider it abandonware. For now it still works at least.

Its GUI is much more capable then LogicPro Scripter, its basically wrapping around JUCE and I think it can probably do some amazing things with enough scripting, but I haven't spent any time with it...and also I question the reason to do so, for sophisticated GUI I recommend just using JUCE.

Except...

Where it might make sense to have more sophisticated GUI options is when you are trying to script Element itself...control Element in a way that a JUCE plugin would not be able to. That is specifically where an Element Scripter would have power to access internal stuff in Element....and would be the only way to do it (similar as LogicPro's articulationID), so then an advanced script-coder might find themselves using Element's scripter and wanting more GUI capability for that. But I see that is not a common case at all needing such advanced GUI capabilities specifically to control Element. I'd rather see Element provide built in modules with their own GUI's...and then just the scripter to control those....and at most, a very simple scriptable custom GUI should suffice. Keeping it simple is useful.

I personally was a little turned off by Lua in some ways, but I know a lot of people are using it....it could be fine with the right custom API

[steveschow]

GigaPerformer

I don't have this yet, but I might get it soon, its on sale right now at Plugin Alliance. It allegedly has a scripter built in and when I looked at its API its was EXTENSIVE with a lot of high level functions. They wrote their own custom language, but the main point is that the API they provide is extensive. You can read more about that here for some ideas:

https://gigperformer.com/the-gpscript-programming-language/

https://gigperformer.com/docs/GPScript36/content/language-manual/introduction.html

https://gigperformer.com/docs/GPScript36/content/reference/list-of-functions.html

I might actually buy this soon because its usually $199 which is a ridiculous price, but Plugin Alliance is OEMing it now at $99 and I have a $20 for the next two days..so $80 to check it out..which is still kind of a lot....but... I am a midi scripting nerd so I might...

[steveschow]

BlueCatAudio PlugNScript

This is pretty decent product with a very similar buffer approach as ProtoPlug. It uses a language called AngelScript which never really took off. Its very C++ like though, so you can prototype scripts and later move the code to C++ fairly easily. I like it better then ProtoPlug because its commercially supported. Its not free.

[steveschow]

One other thing to mention is about debugging. None of these provide good debugging capability. This is worth thinking about a little bit. What I have done with LogicPro Scripter is I literally wrote some code in javascript to emulate LogicPro's playback and I can then debug my code using NodeJS and Visual Code Studio. Setting it up is not for the faint of heart. I'm halfway done preparing a gitlab repo with something to share for other people to make it semi-easy to setup visual debugging of LogicPro Scripter scripts. For little simple scripts its rarely needed, but I have gotten into some scripts with thousands of lines of code doing a lot of complicated data structures, and basically when I needed to debug it was a PITA.

So.. whether you're talking about Lua or Javascript or any other script, some thought into how help them debug scripts would be useful. its not practical to try to connect with real visual debuggers. But something more than simple logging facilities would be very useful. Just something to think about.

[steveschow]

I spent some time last night getting familiar with GigPerformers scripting support. Its not very good. Terrible home-grown language with a lot of limitations. There are still a few things to learn from what they did perhaps, but I personally find LogicPro Scripter a lot more capable simply because Javascript is more capable, and the same would be true of Lua. They did provide a lot of higher level functions though. you can read their docs here:

https://gigperformer.com/docs/GPScript35/

[steveschow]

I found the lua plugin to try out in Element now, is there any midi support in there now? The example script is audio related.

[mfisher31]

I found the lua plugin to try out in Element now, is there any midi support in there now? The example script is audio related.

There is, but admittedly not documented yet. the second parameter in node_render is a kv.midi.Pipe which is implemented in C for realtime purposes: https://github.com/kushview/lua-rt/blob/feature/submods/src/midi.c

This is the best I have so far, but the intention is to expand the RT api with more high level functions. There is virtually no documentation yet. a kv.midi.Pipe is an array of kv.midi.Buffer's which has an insert method and gives access to available events. Nothing high level yet.

https://github.com/kushview/lua-rt/tree/feature/submods <-- work in progress to rename the library to "kv". This is the branch used by Element.

Make shift test suite has some midi demo's in it: https://github.com/kushview/lua-rt/blob/feature/submods/tests/test.lua

[mfisher31]

Oh, I should mention: Only realtime Lua bindings will be written directly with C. For application level (e.g. runs in main or non-audio thread) will be bound with the excellent sol3 C++ library.

[mfisher31]

I should also mention, :), I have tried sol3 bound directly to juce objects... no dice for realtime.

[mfisher31]

And yet another note.... sorry! I'm open to binding any language for app level stuff. We'd just need a common system written in the c++ backend that makes the loading, cataloging, and executing of user scripts in a common way.

[steveschow]

Thanks for the insights. i will try to have a look. I don't have any experience with script binding in C/C++, but I am a retired dev and was mainly doing C/C++ back in the day. I would be happy to contribute to ongoing development of Lua scripting in element as it starts to shape up, but will need guidance in terms of how it taps into the existing code base. So once you have the general system finalized, I could contribute to developing API related modules..for example.

Anyway I'll have a look at some point...