Feature request: POSIX signals? Local socket?

[knghtbrd]

Cleaning out some github bug notifications in my email, I saw one related to #4 and rxtx not being properly supported in ADTPro proper at the moment. Let me define the problem a bit and maybe you'll have some thought about how best to solve it.

As you know, the first feature of a2cloud is that we start it "headless" so that it can provide virtual disks even if you're not running X. It does this by … running X, at least a virtual framebuffer server intended for testing. The problem is that our only way of interacting with this running server is to unceremoniously murder it with the kill command and start another copy with changed settings. This causes race conditions where we end up with multiple headless X servers that don't quit because the program that ran them still appears to be running.

We could do this a lot more easily and with a lot less mess if we had some way to tell ADTPro to rescan its virtual disks after we change their symlinks, say. This could be done by sending perhaps SIGUSR1 to the server. I don't know if you could do that in Java. Obviously that code would do nothing on Windows, but on Mac and Linux it would work.

Another option might be a socket. A socket might listen for commands "rescan 1" to "change" the disk in drive 1 by re-reading whatever the symlink points to. This might be the most flexible route because it could eventually be extended to allow adtpro to run completely as a server with no GUI on UNIX and Mac systems with a really simple client command telling the server to say start a bootstrap or possibly even configure or reconfigure serial port parameters.

When I started this discussion, I was leaning toward the idea of the signal if it's doable (and I don't know that it is). For the socket idea, I just don't know if Java supports local IPC sockets—it might have to bind a port on localhost. It's also probably a bit beyond something I could just bang together a PR for you in an afternoon given a pointer to the requisite API with my current skill with Java most likely. Still not even sure if USR1 and USR2 signals are things I can use from Java code, so it might have to be the more complex option.

Do you have thoughts about any of this?

[david-schmidt]

Signals are a UNIX-y thing, and UNIX-like OSes aren't the only places ADTPro runs. So signals aren't really a way to think about this. So that begs the question: what is really needed here?

We used to do bullet-proof open-read/write-close operations on virtual disks so that everybody always stayed in sync. But, largely due to prodding from a2cloud, that was changed to be much looser because the pi couldn't really keep up and performance suffered (a lot). So the reason it is the way it is is because of pi and performance "optimization."

The problem is kind of broad, though: consider a connected client that is operating on a disk. If we shuffle the underlying virtual disk (potentially changing not only the contents, but other aspects such as size) we stand a chance of confusing the Apple II OS or software that is interacting with that disk. Maybe not a huge deal in the floppy sense - users could always remove and replace those at will - but maybe a bigger deal if it's in the context of a larger or assumed hard drive.

ADTPro (obviously) already has an interface to give it commands - it's the protocol that clients talk to it with (http://adtpro.com/protocol.html). We can always add a protocol function to re-read disks or whatever we like, if that's a way to get at the desired result. I'm just kind of fuzzy as to how and when it would be good to ask it to close and re-read the virtual disk handle. Maybe it should just do that every so often on a timer if it's found to be open.

[knghtbrd]

The Pi 2 and 3 can keep up. The Zero and the original A/B/A+/B+ cannot.

What I'm really after is trying to improve one, possibly two things we have right now. The first is vsd, which currently does this:

        rm /usr/local/adtpro/disks/Virtual${drive}.po &>/dev/null;
        [[ ${1:0:1} != "/" ]] && pwd="$PWD/";
        ln -s "$pwd$1" /usr/local/adtpro/disks/Virtual${drive}.po
        [[ $drive ]] && VSD2="$pwd$1" || VSD1="$pwd$1"
        if [[ $(ps aux | grep [A]DTPro) ]]; then
            if [[ ! $skipWarning ]]; then
                echo "Please make sure you're not writing to either virtual drive on your Apple II."
                echo -n "  Press return when ready, or control-C to cancel..."
                read
            fi
            sudo pkill -f [A]DTPro
            while [[ $(ps aux | grep [A]DTPro) ]]; do sleep 1; done
            /usr/local/bin/adtpro-start
        fi

Ivan-flavored bash to English:

1. Delete either Virtual0.po or Virtual1.po, depending on which disk we're changing
2. Create a symlink from a file supplied by the user to replace the file we just deleted.
3. If ADTPro is running, warns the user to stop actively writing to the virtual disk we just deleted. (This is safe to do after because UNIX)
4. Murders ADTPro
5. Waits for it to die (it, but not necessarily the Java that spawned it)
6. (Re-)runs ADTPro to share the new disk

The reason 3 above is safe to do is because UNIX isn't affected by the virtual disk disappearing until the file is closed. Until then all reads and writes are happening to the old file. Since under a2cloud the virtual disk files are actually symlinks anyway, the actual file ADTPro was reading/writing isn't affected at all. The whole reason for killing ADTPro and restarting it is just a way of asking ADTPro to please close the old disk file and then open a new one when it has the opportunity to do so.

The race condition happens when a new ADTPro is started before the old one's Java instance exits, since some optimization or other causes that Java instance to stick around and take over the new ADTPro, spawning a new Xvfb. I've almost never seen this happen on a Pi 3, but did see it on the Pi Zero.

If we could simply indicate to ADTPro that this disk change action is desired, it could synchronize everything, make the disk "not ready", close the file, open the file again (possibly a different file now), and then make the disk ready for read/write again.

Since the use of signals is out, that means a socket. Using a socket would also permit other commands such as telling it to bootstrap the ADTPro client, allowing for truly headless operation as a server.

[david-schmidt]

How about looking at this a different way: since java is the culprit, why not kill the java process that underlies ADTPro? A little more grepping of the process list will give the ppid (java) as well as the pid (ADTPro).

[knghtbrd]

The reason why we kill ADTPro is because (we assume) there won't be more than one ADTPro process running at a time. I've also considered creating a simple Xvfb service file that runs a single instance of Xvfb that exists independently of whether or not ADTPro is connected to it or not. When ADTPro dies, Xvfb won't, but ADTPro can simply connect to the same Xvfb when it restarts. That'd solve the immediate bug. But it still means that if you want to bootstrap an Apple II, you can't do it over ssh or via web interface. I eventually hope to be able to change virtual disks via web browser (or smartphone…)

[david-schmidt]

That’s what I’m saying - dig out ADTPro’s parent java process ID (ppid) from the process list, not EVERY java process. That will have your desired effect.

[david-schmidt]

Closing due to inactivity.