multithreading

[osch]

Hi Stefano,

while playing around with MoonFLTK, luajack and lualanes, I came to the problem, that it seems not possible to create a jack client in another thread using luajack. I tried to run the gui on the main thread, to launch other threads with lualanes and to create a jack client in one of those threads. With this construction - I thought - it could also be possible to have more than one jack client in one application. Howerver this didn't work :-(

But I think, that it could in principle work. I will investigate this further and try to make a proof of concept to evaluate how much redesign in luajack is necessary.

What do you think?

Best regards, Oliver

[stetre]

Hi Oliver, I'm not sure it could work, but if you want to investigate, go ahead :-)

As far as I can remember (it's been a long time since I wrote this...) what makes multithreading tricky in LuaJack is that threads are actually created by the JACK implementation. Moreover, if I remember well, JACK also has some degree of freedom in its threading policy.

The application has to deal with it, which lead me to the architecture I described here for LuaJack, which basically reflects the architecture used in the original JACK examples.

The basic idea is that you create clients from the main thread where you also execute the main loop, rely on JACK to multithread them (you really have no other choice in this, since JACK would to it anyway), and possibly create so-called 'client threads' (the terminology here is confusing) where to execute the GUI or any other non-RT processing. If I recall correctly, they were added to JACK specifically for these purposes.

To sum up, JACK is already by itself a threading library. Not a general purpose one, but it is a threading library. What about mixing different threading libraries and their different policies? Thrilling, to say the least ;-)

[osch]

Thrilling, to say the least ;-)

Yes, I love this kind of thrill :-)

I see that the jacklib contains some threading functionality. But it's not a general multipurpose threading lib... So it could make sense to combine it with other threading libraries. Today's luajack global main thread would become some other jack main thread, i.e. more than one luajack main thread would be possible. For each of these luajack main threads the architecture would be as today with jack process context and jack client threads as in your description.

I'm also not 100% sure if this will work, so it remains a suspenseful task....

[stetre]

You really like threads, don't you? :D
I personally don't like them... I more or less resonate with what Roberto Ierusalimschy wrote on the subject in "Programming in Lua"....

[osch]

You really like threads, don't you? :D I more or less resonate with what Roberto Ierusalimschy wrote

It's not the threads by themself that I like ;-) I also agree with Roberto and think it's a very good idea that higher order stuff like multithreading (or object orientation) is not implemented in the lua language itself but can be build upon lua core. I very much like the effective minimalistic design of lua that provides the basic building blocks.

As I wrote: I'm trying to develop some example code for evaluation how other multithreading libraries could go together with LuaJack. In the first step I did however not involve other multithreading libraries. The currently provided example code shows how multiple jack clients can be created (and destructed) from one gui thread using MoonFLTK.

You can find the example code in the experimental branch under https://github.com/osch/luajack. The example can be build by invoking the Makefile (however cmake is also needed). The example can be found under examples/gui_thru.lua. With this example you can create multiple JACK clients. Each client has input/ouput ports with a mute button. Each client can also be activated or deactivated by button. For each client a separate window is created. When a windows is closed, the corresponding client becomes eventually garbage collected and is automatically closed.

I would love if this functionality could be included into LuaJack. However this seems more work than I expected since it would be a major re-design. I am trying to implement more functionality for the experimental branch (next thing would be to demonstrate the usage of a separate jack main thread using lualanes and also error handling for the JACK process thread).

What do you think?

Best regards, Oliver

[stetre]

I will take a look at your experiments, but still I am not quite sure I understood what you are trying to achieve. Is it really worth the effort to tackle such use cases (multiple clients with multiple GUIs) at the thread level instead of doing it at the process level and leveraging the client/server architecture of the JACK system?

[osch]

Is it really worth the effort to tackle such use cases (multiple clients with multiple GUIs) at the thread level instead of doing it at the process level and leveraging the client/server architecture of the JACK system?

That's a good question... it's not the special use case that I am after. These use cases (multiple clients, clients closing when garbage collected etc.) just are corner cases to ensure that the whole construction covers a broad scope of use cases and provides flexible basic building blocks. It's the JACK API itself that gives the freedom to create as many clients as you want from any thread as you want (at least I got the impression that this is possible). So it feels just natural to have the LuaJack API giving you a similar set of possibilities. Yes you are right: a lot of applications could be realized with the given set of current LuaJack's possibilities. But as I said: it would feel more naturally if LuaJack wouldn't impose a special thread layout for the calling application.

So it's more or less a question of architectural aesthetics. But you are right: it would be a lot of questionable effort to achive this goal. However I will explore this question a little bit further and I'm hoping that you also find these ideas interesting.

I pushed another example to show how jack clients can be managed from a separate LuaLanes thread, see examples/gui_thru2.lua.

[stetre]

It's the JACK API itself that gives the freedom to create as many clients as you want from any thread as you want (at least I got the impression that this is possible).

I am actually not sure about this. Back when I wrote these bindings I struggled a lot to understand how threads were used in JACK and how they were supposed to be used/handled by JACK applications, and after fiddling with the examples and reading discussions on the JACK forum I came to the conclusion that this thread layout is more like 'standard' than 'special', and that other layouts are more like 'freak' than 'general'. Of course I may be wrong... to sort this out it would be useful to find examples of JACK applications that use different thread layouts than this. I think it is important, however, to keep in mind that threads here are used not because they are a nice general construct, but because they are needed for a very specific problem: interacting with hardware that imposes real-time requirements (ie time constraints), while possibly doing other non-RT tasks, concurrently.

I'm hoping that you also find these ideas interesting.

Yes and no. Yes, it interesting from an intellectual point of view. But from a design point of view, as I wrote last week, I resonate with Ierusalimschy's view which is not (provided I understood it correctly) to consider threads as higher order stuff but to consider them as lower level stuff that is not a good idea to bring up to the Lua level. My first attempt with LuaJack was actually to hide multithreading completely, and aim at providing bindings for single threaded scripts with no support at all for multithreading at the script level. But then I realized that it was not possible. Or at least that I was not able to do it without adding another layer of abstractions, which I didn't want to because then they would have not been 'bindings'...

[osch]

I struggled a lot to understand how threads were used in JACK

I'm also wondering for what the jack client threads (e.g. jack_client_create_thread) are supposed to be used. It seems there is nothing special about these threads, except that they are somehow tied to a specific jack client and that you can turn on realtime priority for these threads.

Reading the examples provided by LuaJack I got the impression that you must use these client threads, e.g. for running gui stuff etc.

But it turns out, that my conclusion is not true: I pushed an example on my LuaJack fork master branch (see https://github.com/osch/luajack/blob/master/examples/gui_thru2.lua) that shows, that in current LuaJack you don't need a client thread for the gui at all. In this example the gui is running in the main thread and just creates and manages a client. Of course the client has the process thread which is necessary for real time processing as you pointed out. This constructions seems more natural than the provided examples where the gui is running in a separate jack client thread after the client is constructed. Perhaps you could integrate this example in LuaJack for illustration?

There also seems to be some confusion on the jack mailing list. See e.g. http://jack-audio.10948.n7.nabble.com/new-release-time-td13801i20.html#a13821 where someones asks:

My question is, why does the client need to use a separate disc thread explicitly? I thought that Jack process() callback ran as a separate thread anyway

and someone answers:

Yes, you are absolutely right! The "disk writer client" (I guess you mean capture_client.c?) use a confusingly complicated setup. It would have been much simpler to write to disk in the main thread instead.

I'm now thinking that the client threads are only for special realtime purposes (which is of course not the gui) and that perhaps for using Jack over lua the client threads are only useful for very rare use cases. Or am I missing an importent aspect?

[stetre]

Uhmmm... maybe you are right. Actually, besides the gui_thru.lua example the only other example where 'client threads' are used is midi_dump.lua, which is a port of an original example where the 'client thread' was not even used. I honestly don't remember why I implemented it that way, maybe just to be able to wait on a condition variable, who knows... (my past self should have written a few more notes on these matters).

By the way, the less threads are needed, the happier I am :) Would you like to open a pull request so I can merge your version of the example?

[osch]

Would you like to open a pull request

yes, of course, see #4 .