Framework to run and keep track of processes independently from ZPUI

[CRImier]

As I've been working with Raspberry Pi, there often was a need to run a process for a long time - say, a mjpg-streamer instance that'd stream video from a USB webcam to the Internet. I was, at one point, working on an mjpg-streamer app for ZPUI, so that I could set up a stream from the USB webcam with a button press. During the development, though, I understood that restarting ZPUI would either kill the mjpg-streamer process or orphan it (don't know which is worse). A solution for that would be having a separate system that'd have some kind of RPC interface, and would:

1. Start and stop processes using RPC (passing the command-line through RPC from the process that ) 1.1 Optionally, add a restart-on-failure feature
2. Keep track of running processes, expose their list to RPC clients, so that they can then act
3. Consequently, allow adding some kind of tags, so that RPC clients can recognize processes that belong to them (and previous instances of same clients)
4. Expose stdout and stderr to RPC clients 4.1 Optionally, allow injecting stdin
5. Keep track of processes that have finished execution, preserve output and exit codes

Now, this would be helpful for many different ZPUI apps which have to launch a long-term process and keep track of it. Questions from me:

1. Are there any existing solutions that we could reuse for this? Does something like systemd (which current ZeroPhone images are based on) offer facilities that we could reuse - in case of systemd, that'd be something like "dynamically created non-persistent units " -, and what part of needed functionality do those facilities offer?
2. What would be the security model? Do we need to protect somebody against something, if so, what is it?

Useful links:

• Systemd user service management - could be used as a backend?

[olafura]

I feel like this is one of the things that can tank a project like this. If you really want control and easy management I would recommend https://nerves-project.org/ and https://github.com/esl/Pyrlang

[CRImier]

@olafura thank you for your suggestion! I've read your links, and I unfortunately can't exactly draw a parallel between what we're looking for and what you suggested. Could you maybe show how we could solve one of the described usecases, or achieve the majority of stated goals, with one of the two solutions you offered? Sorry, it's my lack of understanding based on the description of the services you've shared, I feel like a link to a tutorial of some kind would clear the confusion up.

Also, do keep in mind that a Pi Zero isn't actually capable of running a true VM (neither in terms of CPU extensions nor CPU time).

About "tank" - that's a fair point, and this isn't actually on the roadmap yet - just a future problem waiting for solutions, in case somebody is interested helping ZP (like you are doing =) ). I'm also thinking about the usecases described, and looking for a way to have them implemented without having to actually achieve the goal of this issue - for example, with mjpg-streamer app, splitting apps from ZPUI into separate processes (and thus avoiding the restart together with ZPUI) is going to solve the described usecase for the most part.

[Fnux]

I'm familiar with nerves, and it's not what we need: it uses a buildroot base image and run BEAM (the erlang VM, similar to the JVM for java) instead of a standard init system. The ZP is based on a full (derived from debian) distribution, which is ways more complex and need a proper init system (systemd for raspbian).

From my point of view, the best way is to rely on existing and standard tools. We should be able to build what @CRImier described with systemd and dbus.

[olafura]

@CRImier What a phone needs is to be able to manage a lot of different services and have a good response mechanism for it, systemd isn't really designed for those kinds of interactions. Dbus is better but it's more complicated than something like GenServer and Supervisors in Elixir. Erlang was designed for phone services and most of the phone calls and router traffic is actually managed by it.

@Fnux The JVM is a totally different beast, Erlang VM is really memory efficient and light. It's way faster than Python so you shouldn't have a problem. You don't need a proper init system, you have to manage everything yourself if you want it to be reliable it's not a server.

I own a couple of Pis including the Zero so I know what they are capable of. I find them quite sluggish with Raspbian but that isn't not the point of the distro, I like that I have access to all the debs but I would not want a phone that had a full distro, except for maybe testing things out.

Personally I would go with Nerves + Qt/QML + snaps for apps(https://snapcraft.io/). But it's your project I'm just trying to be helpful. Because what really kills these kinds of projects is two things, managing services and apps, and I've probably monitored most of the projects that have tired something like this. I almost got hired by a company that was going to write python services and use Firefox phone for the interface, but they ended up not being able to hire anybody.

[Fnux]

I only compared BEAM and the JVM with from the "execute bytecode" point of view, I know they're totally different ;-)

I know @CRImier uses its ZP by just SSH'ing on it, and it's nice to have a full distribution in that case. Given the target (DIY phones), I think a full 'hackable' distribution makes more sense. Moreover, the current UI is written in python.

[olafura]

The problem with having a full distro for anything but developing and testing is that the boot times are horrible, you can also ssh into nerves but you don't have the same access to install random stuff. Except if you enable snaps.

[CRImier]

@olafura that's quite an insight. I understand your idea and suggestions better now, I'm going to think about it some more - for now, here's the "why a full distro" response, hope it gives you some insight.

---

About having a full distro - it really adds to the hackability, which is a priority. ZP is a platform where I want to give users a chance to improve their own phone, even if they don't have a machine with a development setup. For that, I believe, having a full unrestricted distro is going to be important, including installing random stuff. For example, if a curious ZP user wants to try out an IR temperature sensor, they should be able to connect it to exposed I2C pins, launch Python, experiment a little bit, make it work and then make an app around it, following a tutorial. Or - make a small script-service which logs temperature by default, and also has an RPC API so that you can take a peek at the temperature through the app. Or, instead, write a lockscreen plugin displaying the amount of money in their Bitcoin wallet. I want to enable people to do stuff like that, even if they only really have a Windows laptop and cannot install a development environment.

What about boot times? Currently, I don't view decreasing boot times as a priority (though I'm sure there's some low-hanging fruit there we could pick). Currently, my ZP has 2 day uptime - and it's only 2 days because it randomly rebooted for some reason 2 days ago, I almost never shut down/reboot it. That's the usecase - given that RPi doesn't have real sleep/low-power modes, it makes sense to keep the phone booted up all the time. With 20h uptime on stock batteries (and no real power management improvements yet), what if the boot time is 20-30 seconds, if you just connect the charger each evening and disconnect in the morning?

What about responsiveness? For now, ZPUI interface and apps are very responsive, which makes sense - work is only ever really done when a button is pressed, and there's usually not a lot of code involved when that happens. I hardly ever see ZPUI consuming more than 1% of CPU in top - when there's a task in foreground that's calculating something and updating the screen each, say, 1 second (for example, the TOTP app I just checked), it can rise up to 7%, but that's rare (and I'm going to expose primitives so that people can optimise apps running in background - some apps already do that).

So, it seems your experience makes you think this is not a good approach. It's obvious that I'm not seeing something you know, so I can't want to hear your insights.

edit pressed Send without finishing, my bad.

[CRImier]

However, there's also something I need to state - I do care about stability, and I do prioritize being able to use a ZeroPhone as a daily driver - for that, the software&hardware needs to be stable. I also do acknowledge that, no matter how I market the ZeroPhone, even if it's "WIP phone with no software guarantees", people will expect and care for stability, and it's unlikely users will advance from usage to tinkering if ZP is not stable as a phone. I also do know that, as you expand the codebase and allow easily making custom apps, plugins etc., there's more potential for errors => instability. I just believe that entry barrier from using ZeroPhone to developing with ZeroPhone should be as small as possible, and it seems to be doable so far - for example, we're covering ZPUI with tests slowly, function-by-function, and it's helping avoid errors when new functions are being added or code is being refactored.

Is there more potential for errors when you have a full distro, too? Yes, absolutely - for me, it's like OpenWRT vs Debian, where OpenWRT is significantly more tailored to be a base for a home router, but using it in an unintended way isn't always easy (in my experience), and getting a toolchain to work can be a major hurdle (again, experience), whereas Debian might not be as tailored, but you can absolutely tailor it to whatever task you want to accomplish, and you're not really limited in what you can do.