feature request: callbacks still needed

[mfisher31]

A callback is needed for for the following things:

• reset (or whatever name you want), I believe there is a parallel with VST/AU yea? in LogicPro this callback is called when the transport is started and when tracks are changed from disabled to enabled. I think different daws might deal with this slightly differently. But in any case we need to run some stuff when playback is started and possibly some other situations such as activating the track hosting Element.

• latency - way for the host (element) to ask for the Lua script reported latency. This could also be put into the layout structure I suppose but there are use cases where the script might be updating the value as it is used, so Element needs to check for that before playback.

• idle - an idle callback would be useful for putting non-realtime-thread operations someplace on lower priority, etc. logging and tracing for example. I presume the UI scripting is handled that way off the realtime thread, but we might want to put other tasks out of realtime thread...maybe that can be done in the UI section, not sure.

[mfisher31]

@steveschow - also relocated this one.

[mfisher31]

• idle - an idle callback would be useful for putting non-realtime-thread operations someplace on lower priority, etc. logging and tracing for example. I presume the UI scripting is handled that way off the realtime thread, but we might want to put other tasks out of realtime thread...maybe that can be done in the UI section, not sure.

Not sure how to go about this one. Thread sync + garbage collection

[steveschow]

Well anyway maybe it doesn't matter because the ui script could handle that off the realtime thread right?

[mfisher31]

It could indeed. I guess it depends too on what kind of data needs shared between DSP and UI. Been trying to think of a way to bridge the gap. Could do thread locking if the entry point is always known, like in an idle() function, but even then we run the risk of things getting deleted in the background while locked and causing drop outs.

[steveschow]

I don't like the idea of locking threads, etc. The JUCE way using OOP design patterns to communicate between the gui thread and the dsp thread, is probably how idle might be used. In truth, it may not be needed all that much, its handle in LogicPro, for example for detailed logging to the console, you want that done outside the realtime thread. so in LogicPro, you can simply fill a buffer from the realtime thread, and the idle call back is used to flush the buffer to the console at a more snails pace without affecting the real time thread. In logicPro the idle callback can be used to do gui manipulations, which in Element would just be in the UI thread to begin with. anyway, since you have the UI thread already, that may be good enough, or perhaps you need to have the printing to console already make use of the UI thread somehow and that will cover the need.

[mfisher31]

Thread locking is not good indeed. Even with an idle function on the DSP script, thread locking would still have to be used. This is because said idle() would be executing in the same Lua context (the script) as process() and therefore need a lock.

More and more I'm thinking add idle() to the UI, or do a "port event" callback similar to LV2, namely lv2:Atom. With a few new lua helpers writing Atoms to an output Atom port would be cake. UI could then parse them in the port event callback... just like LV2. 100% thread safe and zero locking when implemented correctly. LV2 hosting in Element already has code for all this that could be API-ized for use in the DSP script.

LVTK could provide all the atom writing, and would be a great addition to it. https://github.com/lvtk/lvtk

LV2 Atom specs https://lv2plug.in/ns/ext/atom

And BTW, if you ever wanted to, from the ground up, get in on a Lightweight GUI library with performance in mind... definitely check out LVTK. I've made some good headway recently. The base library is geared toward LV2, but all of it, including GUI can be used in anything. The build is simple with meson for Linux/Mac/PC/*nix

[mfisher31]

Yeah, LV2.... even the latency thing could be exactly like LV2.. have a "latency" designated output control port (parameter). And the host, Element, just reads that and updates the engine like any other node.

Yep, not done with the Script node quite yet for 1.0. Using parameters like this need a "hidden" property in the layout... again like LV2. That stuff is baked in my mind I think..... haha

[steveschow]

Ideally it would be good to be able to set the reported latency using script code to do it, because it can be different. For example, one scripting thing people will want to do in the community I participate a lot in, is using scripting to compensate for orch sample attack latencies which are inherent in orchestral sample libraries...and change from articulation to articulation. So they need scripts that can detect which the current articulation and create a negative delay offset to compensate. That essentially means creating lookahead latency so that you can send each note on a per articulation basis some amount early...and the lookahead will be reported to the host. Each orchestral library will be different and ideally the script will have either ra bunch of Lua tables hard coded, or could read JSON or XML files to get it, and then compute in the script how much lookahead to configure, etc.

So anyway, the point is, that it would be very helpful to be able to report the latency, using LUA script code. In some other script plugins out there, they essentially have a callback of some kind, and the LUA can respond to the callback. Which is I think how JUCE works too yea? Anyway, that way the code itself can decide what latency to report, which the host will check every time you hit play before beginning to play..or something like that...

If we have to configure latency reporting outside of the LUA, it will be much more fiddly and the user of a script will have to know how to determine what the right amount of lookahead is, or the script would have to depend on a fixed lookahead amount, even if its way more than necessary, etc.

Anyway, the fundamental need there is simply that the LUA code can in some way set the reported latency, and do so a bit dynamically...not in the middle of playback, but certainly not once per instantiation of the script node, etc. The script needs to be able to set it perhaps for each time the transport is started.

[steveschow]

Nice thing about element for the above use case, will be that someone can deal with orch sample library peculiarities...using lua...and also sub-host the instrument inside Element, which gives sample accurate midi filtering... so it really has a lot of potential for this kind of task.. People in that community today or doing the tkind of stuff using the script engines built into Kontakt and stuff like that, but Kontakt's KSP scripting pales in comparison to Lua, and of course many sample libraries aren't in Kontakt...so I see element as being very useful for this kind of task and a few others....once the Lua rollout is fully stable I will be recommending it and some sample scripts.

[mfisher31]

Hey @steveschow, what's you're take on a "headless" UI? Like a property returned in the DSPUI script indicating as much... I'm thinking that way idle() can be used separate from DSP but not necessarily have a custom UI.

[mfisher31]

Oh and my idea to use LV2 tactics for latency is inside Lua, not outside. Instead of a function callback, you just set a property with a value, then Element at the C++ level sees that and updates the node just like any other node.

[steveschow]

I guess by headless you mean no gui? Not sure what the question is. I have not tried to work with Element's lua approach for working with plugin parameters yet, so I'm not sure what you have done already. I will get to that eventually. I can provide some commentary about other scripter plugins I have used (LogicPro scripter, blueCatAudio PlugNScript, Lua ProtoPlug).

First about LogicPro, which I am the most familiar. LogicPro hides internally a lot of what it does, in order to make it an easy scripting APi. I suspect some tasks are done in the realtime thread, as callbacks. and some might not be. For example we talked about the Idle() callback they provided. In truth, hardly anyone uses that callback, almost everything has to be done in the realtime thread in the end. what I have used it for was for flushing a console logging buffer because the built in console logging behavior would only output so many lines and eventually might say "sorry too many lines in the realtime thread' or something like that. But I was able to make a buffered logger that would simply stash the logging into the javascript equivalent of a lua table, and then later on during the Idle() callback I could finally send it all to the console window. The logging would usually fall way behind the actual midi traffic because Idle isn't called as often, etc, but at least no message would be left unlogged and the realtime thread would be less burdened also.

In respects, console logging is like a GUi operation and should really be part of a separate gui thread, but LogicPro scripter doesn't provide a separate script for that sort of thing like you are doing with Element. You basically just provide known call back functions and it calls those on whatever thread it has decided makes sense internally to the Scripter plugin.

In any case, there are a few other use cases for the Idle function, but generally its only because Scripter isn't providing its own built in functionality, such as proper console logging on a gui thread, or perhaps something related to default parameter gui. Most people don't use it and don't understand it. And even me personally, probably don't care about it as long as console logging is happening off the realtime thread, for example, or anything related to gui stuff, which you are already attempting to do kind of the JUCE way, but maybe without so much of the JUCE OOP design pattern, I'm not sure yet I haven't gotten into that yet to see what you did. But in any case, I do think it would be generally advisable to make the scripter much mor simple then the complete JUCE way of managing messages between threads or whatever, I can't remember now there was the old JUCE way and the new way, I haven't gotten into that yet. I think since you have UI thread anyway as part of it, don't worry about an idle callback, but try to make sure Element is already taking care automatically of things that people might have tried to use Idle(), such as the console logging I mentioned, anything related to GUI (which presumably you're already doing), and perhaps some things related to parameters.

Regarding parameters, LogicPro scripter does not have the advanced gui capabilities that you are adding to Element Lua. It basically just lets you define a list of parameters in a javascript equivalent of a lua table, you defined each parameter, the type of parameter it is (menu, checkbox, momentary, slider), some default values and data range, specify whether it should automatable or not, and whether it should be hidden or not, etc. A few different things. Then an automatic default simplistic GUI is shown for those defined parameters. In addition they will show up in the host as automatable parameters unless you specifically disabled that.

Ok so far, simply define the table in the script and there is the GUI and parameters present. Once they are present, the serializing step (saving and loading) is handled completely automatically, the script doesn't have to do anything. Also, the simplistic gui is automatically there, automatic. No scripting needed unless you plan to dynamically move things around on the GUI, show/hide elements, etc... (which is one good use case for Idle).

the script can directly access the parameter value with a simple function like GetParameter(5). which is fine and easy to do, but it turned out that this operation was very expensive computationally in Scripter. I'm not sure if it's because of locks or what, but I wrote a blog post about it, and found it to be literally 60 times more time taken to retrieve a value from GetParameter compared to a simple assignment. You can read that blog post here for reference:

Here

The work around was to use a callback provided by LogicPro whenever a parameter value is set or changed via the default GUI or via host automation. Inside that callback assign the new value to a global and then the other realtime callbacks always just use the global and don't bother to look it up. This is unfortunately probably not sample accurate, but for midi, probably better, because it was hitting the CPU pretty hard if you called GetParameter in every process block. See the article above.

In the case of PlugNScript, they handle it more automatically, I can't remember if they have. parameter changed callback or not, I would have to look it up, but I think you just declare some parameter globals at the top of the script and they are kept up to date automatically behind the scenes without the script having to worry about it. Also I am remembering off the top of my head that they also pass the current parameter values as arguments into the process() callback, which must ensure they are reasonably close to sample accurate in terms of timing what those values are supposed to be during the process block period of time. That is much better then Apple's approach in LogicPro IMHO.

Anyway, then the script can just get the parameter values very inexpensively and very easy and serializion of the parameter values is automatic, as is a simple default gui without the user having to code anything for it. If there is synchronization between threads, the Plugin hides all that behind the scenes. Might there be some locking somewhere? I would say possibly, just as JUCE might be under the scenes, though good design patterns can reduce that possibly eliminate it. But none of that needs to be exposed to the script.

anyway, there is a lot of overlap between GUI and parameters. the only difference is that parameters are meant to be important pieces of data that need to be saved with the project when you save the DAW project, all the current parameter values should be saved also, so that when you load up the same project again later, all the plugins have their current state as saved brought back. These scripter solutions just make all of that very automatic for default cases of specifying a list of parameters...and they make it very easy without having to get into multi-threading design patterns, to access the parameter values via script code, even set the values if you want.

i don't know if that is making sense...but eventually I will come back to trying what you've done so far with Element's handling of parameters in lua, right now i want to get my EventChaser juce version done and preoccupied with life at the moment too.

[steveschow]

Oh and my idea to use LV2 tactics for latency is inside Lua, not outside. Instead of a function callback, you just set a property with a value, then Element at the C++ level sees that and updates the node just like any other node.

That sounds perfectly fine, as long as we can set it in lua and change it also (not during playback of course)

[mfisher31]

Yeah, definitely get in there and have a look when you can. Element is doing most of these things already, namely parameter access in DSP and UI, and auto-saving paramters defined in layout(). I made it so print() in lua prints out to the UI console in the Gui as well. Think I'll just put idle() on the back burner. Usually that kind of callback is needed to drive an event loop, which element doesn't really need since the Lua UI on the backend is driven directly by juce.