Use of pidfiles

[grammoboy2]

While I was at it on the bashguide, fyi:

"Using PID files is risky".

https://mywiki.wooledge.org/ProcessManagement?highlight=%28sigkill%29

If you've two nsmd servers running, they could in theory communicate with each other I would think. Then there is no need for a PID file..

Nsmd stores the pids of the clients? So, according to that document, that's (pretty) reliable if I understand it correctly.

[grammoboy2]

Also, I wonder what the implications are for using the XDG RUNTIME DIR, which seems to be a special one in the XDG family.

"The lifetime of the directory MUST be bound to the user being logged in."

I wonder if this affects network sessions. Also, when I logout, nsmd seems to be running still. Anyway, things to think about at least.

" $XDG_RUNTIME_DIR defines the base directory relative to which user-specific non-essential runtime files and other file objects (such as sockets, named pipes, ...) should be stored. The directory MUST be owned by the user, and he MUST be the only one having read and write access to it. Its Unix access mode MUST be 0700.

The lifetime of the directory MUST be bound to the user being logged in. It MUST be created when the user first logs in and if the user fully logs out the directory MUST be removed. If the user logs in more than once he should get pointed to the same directory, and it is mandatory that the directory continues to exist from his first login to his last logout on the system, and not removed in between. Files in the directory MUST not survive reboot or a full logout/login cycle.

The directory MUST be on a local file system and not shared with any other system. The directory MUST by fully-featured by the standards of the operating system. More specifically, on Unix-like operating systems AF_UNIX sockets, symbolic links, hard links, proper permissions, file locking, sparse files, memory mapping, file change notifications, a reliable hard link count must be supported, and no restrictions on the file name character set should be imposed. Files in this directory MAY be subjected to periodic clean-up. To ensure that your files are not removed, they should have their access time timestamp modified at least once every 6 hours of monotonic time or the 'sticky' bit should be set on the file. " https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html

[diovudau]

I want to use this opportunity to tell you about "network sessions", that were after all announced as very special and "unique selling point" by Jonathan and the NON website over the years.

It was a marketing trick all along. There is no communication between nsm daemons or processes. There is not a line of special network code in the nsmd source.

"Network sessions" was purely a feature of the FLTK-GUI that connects to an already running(!) nsmd on another computer over OSC. Given that OSC is already a network protocoll there is little special about this left. It is especially not a way to start nsmd servers over the network and then load a session there.

One could just use the nsm url environment variable or command line parameter and connect to another remote nsmd through that, given the network access of course.

The only thing special in the FLTK Gui was that if you connected to a remote nsmd the GUI would ask you where to start the program, locally or remote. That is useful enough and still available of course.

Bottom line is that I was tricked by that as well and thought this "network session business" was some big thing and I had to read the code to understand it and then trouble implementing that in Agordejo as GUI. Turns out there was nothing to fear.

Edit: With original NON-SM you would even need to know the port number beforehand. There was not autodiscovery. So either try your luck with a fixed port nsmd startup during boot or have a look on the 2nd computers screen yourself. Server discovery, and seeing if there is a session already running or if the server is in idle-state, is what the new-sm update implemented. Those are not GNU PID files btw, that use special operating system features. It's just text files in a dir that garbage collects itself, so a power-outage will not leave a lockfile around, even if there is no server running (that was previously the case)

[grammoboy2]

Ok, but that dir has special properties it seems, like the possibility to get removed after 6 hours (did I got that right?) and being removed when logging out. That doesn't sound ideal for a nsmd situation.

I'm wondering, there is a server running. Then why is a (lock/pid/state) file needed to get to know which session is currently loaded by a certain nsmd server?

I can imagine some communication line between running nsmd servers, which tell each other what they're doing. Broadcast messages between servers so to speak.

Related to this, it's currently impossible to ask nsmd which session is loaded at the moment as far as I understand. I can imagine it would be useful for a external script or tool (maybe even live coding language) to be able to ask the server for information. Hey, which session do you have loaded, which clients are running, what is the status of the clients?

Currently I lack the technical knowledge to think this out. I can imagine though, these things will add more complexity, you don't only have different servers, but also different GUIs. So these are just thoughts I've, which might be not the best ones on a second thought. A modular setup has already lots of complexity (strong as the weakest link) and a strong part of NSM has always been to keep things simple and straightforward. Ideally there is a method which always works in all situations. The original lock files didn't apparently, so files might not be the right method. Also ideally it would be backwards compatible, ideally the original Non-Session-Manager GUI would still work (which it doesn't with the forks server it seems, I don't know why nor see a reason why it wouldn't).

[grammoboy2]

The problem of Non-Session-Manager seems to be what is called a 'stale lock'. There is still a lockfile, which shouldn't be there anymore. Flock might be as well a simple solution for it. https://mywiki.wooledge.org/BashFAQ/045?highlight=%28flock%29 https://unix.stackexchange.com/questions/594027/handling-of-stale-file-locks-in-linux-and-robust-usage-of-flock

Not sure how this can be implemented in C++. Via https://linux.die.net/man/2/flock ?

I can imagine a lockfile in a directory is useful if you also want to block access from let's say a filemanager. How this actually works and whether that is desirable or not, I can't tell at the moment. All these things need a lot of thought and testing, advise from Unix experts etc. before it can be implemented I think. It's beyond my expertise atm at least. My 2 cents.

[diovudau]

I am not sure about which problems you are talking about. Every problem you mention here has already been solved and implemented. Have you actually tested any of it?

[grammoboy2]

The missing lockfiles when creating and duplicating a session, are probably explained by the fact that the Non-Session-Manager might not had a refresh button in early stages. As I understand it, the lockfiles where designed to prevent a other nsmd/ GUI on the same host from accessing the session. If that other nsmd created a new session, the other was not able to connect it, it was not visible in the GUI.

Accusing someone from false marketing is not fair here. The fact that it uses OSC contrary to other session protocols at that time and the fact that a single GUI could manage two servers on different hosts, makes it a fair point to make on the website. Also, simplicity is not a bad thing.

[nedko]

It was/is false marketing. ladish was able to do multihost network sessions in 2009 [1]. According to NSM [2] site, "In 2010, shortly after the release of Non-Mixer, I" (Jonathan) " devised a prototype version of the NSM OSC API and have been using it, still unreleased and in prototype form, since then."

[1] https://lists.linuxaudio.org/archives/linux-audio-dev/2009-November/024375.html [2] https://non.tuxfamily.org/wiki/History

EDIT: ladish support for network sessions is actually rudimentary, but uses TCP instead of UDP and has no issue with pid files, as dbus-daemon handles this. A systemd unit for nsmd may help some.

[grammoboy2]

Hi Nedko! You've done a lot in the realm of session management in the linuxaudio world. Ha, nice to hear from you again.

False marketing implies bad intentions. I still don't think this is the right word choice here. Also following quote contradicts your statement somewhat, from the same LAD conversation:

"Multihost ladish is planned for the 2.0 release and wont happen anytime soon unless someone with high motivation steps to work on this."

Latest ladish release I found was version 0.3.

Note, my mentioning of pidfiles, was because I read about it in the bashguide. I'm sure it works in most of the cases for new-sm, just not sure if it's the best design. Reading about XDG_RUNTIME_DIR, fueled this doubts too. Systemd doesn't sound very attractive to me either though.

[nedko]

Latest tarball release [1] of ladish is now 11 years old (maintenance is provided by publishing git commits, as was the case with development in the times of [2]). The multihost capabilities (to the rudimentary extent that ladish has them), have not changed for the last 13 years.

I'm working on a project (jack plugins) to provide a launcher that will implement the relevant parts of the NSM protocol. The launcher is designed to be snap-in replacement for nsm-proxy. A future adoption of jack-plugin-launcher (jpl) in NSM protocol managers (as nsm-proxy replacement), projected as possible technically, could solve lot of liblo related issues, including but not limited to this one, of NSM setups and thus benefit the users.

[1] http://lalists.stanford.edu/laa/2011/12/0007.html [1] https://lists.linuxaudio.org/archives/linux-audio-dev/2009-November/024375.html

[grammoboy2]

We will wait and see what you come up with. Can we find it somewhere?

Solving a lot of liblo issues, that sounds you're using dbus or something related, e.g. not OSC?

But if the problem is liblo, then liblo should be fixed, or there must be a C/C++ OSC library without those issues.

[diovudau]

I am still uncertain what the original problem of the issue seems to be. We are diverging into offtopic, and before I have to close the issue I want to ask for a rephrasing of the issue or bug again, please.

[nedko]

@grammoboy2 I seriously consider implementing the fd:// address scheme for OSC communication in a dedicated lib that should be at least as ease to use as liblo or even easier. In a long term though, improving libosc makes sense. That would be a next step after implementing jack-plugin-launcher with the dedicated lib (I name it liboscpipe).

@diovudau You are correct, pidfiles are not strictly related to multihost capability probably, an github issue where such comments will be more on-topic will be more appropriate. Still autolaunching of daemons (in multihost setups too) in one way or another involves dealing with pidfiles, so this is not completely off-topic, IMHO.

[grammoboy2]

@nedko A replacement for NSM-Proxy, sounds to me like you want an alternative to launch jackd applications in a session environment without the need for native NSM support. I think what NSM did better then Ladish, is to keep keep things simple and clear, without (optimal) support for less then optimal solutions. NSM has basically one level of support, NSM, which is quality guaranteed.

A other thing that NSM did well, is to stay away from Desktop dependent technology. I think a osc library is the only dependency. No dependency on jackd, dbus, systemd or whatever.

So having a more optional 'NSM-Proxy' sounds nice, but I'm afraid the end result will be that we'll have to live with applications that have less then optimal session/ NSM support. Then I would be more in favor to make it more easy to patch NSM applications, for instance for applications like Surge, Vital, which uses the JUCE library.

A other thing that could improve the modular session environment is probably a lightweight LV2 host with NSM support. Carla tries to support everything, which could be nice, but for a modular setup I would rather have a more one-task-one-tool approach, with fast startup time, reliable shutdown etc. Something like Jalv with NSM support would be ideal probably.

[diovudau]

And this concludes our discussion-time. Because there was no actual flaw in existing code described and discussed this issue will be closed now.