Erlang-style monitoring

[sylvanc]

When an actor is no longer reachable, either because it has been garbage collected or (in the future, on a distributed runtime) a node has crashed taking all the actors on it with it, some other (still running) actors may want to be informed.

I think the best thing to do is copy from the best, and provide an Erlang-style monitoring facility. It's on my to-do list. I'll use this issue to post updates as/when they happen.

@darach and I put together a preliminary notion of how it will work. Since actors are GC'd, notification can't use the actor's memory address as a unique identifier, so when an actor is first "monitored" it will generate a unique non-memory actor identifier. On the distributed runtime, that ID will need to be node-aware so that it doesn't conflict with IDs generated by other nodes (probably using a prefix rather than a more expensive PRNG).

Then, when an actor is GC'd, monitors will receive a notification based on the generated ID rather than the actors (old) address. Similarly, in the distributed runtime, when a node dies, all other nodes with references to actors on the (now dead) node will generate notifications for the set of (now dead) actors.

[andreaTP]

Are you planning any kind of hierarchical model for actors?

[sylvanc]

With monitoring, the programmer could assemble a hierarchical model, but I don't think we want to enforce one. That is, if actor Alice spawns actor Bob, Alice may not want/need to monitor Bob.

Or did you mean something else?

[darach]

@andreaTP I'm thinking of an OTP package to make pony easy for Erlang folk so we can add GenSupervisor there. It could be a separate project or packaged with pony it doesn't matter.

Monitor and Link means anyone with a different use case that doesn't quite fit Erlang style supervision can roll their own. Erlang's model is powerful though, and a sensible ( but optional ) default.

[jemc]

Hierarchical monitoring is cool for a lot of applications, but I agree with @sylvanc's point about not enforcing it as part an integrated part of the implementation. Instead, it could be a package that lets you easily do erlang-style hierarchical supervision on top of the "native" pony monitoring.

[edescourtis]

As an Erlang developer I can say that supervisor trees and proper isolation (resource collection etc..) are MUST haves. They are quite literally the MOST important feature in Erlang. I wouldn't bother with Erlang if it didn't have it and I think most of the Erlang community would agree with me. It would also be a good idea to think about having in flight upgrades because it enables systems that run forever. If Pony doesn't include these features I cannot consider it over Erlang no matter what performance or security advantages it has. My order of priorities is as follows (most important to least important):

1) Works (even in the presence of programming errors) 2) Secure 3) Scale to multiple machines 4) Performance 5) No service interruption during maintenance

Right now from my perspective it looks like Pony has 2 and 4 with the goal of supporting 2,3,4. To consider it for production I need 1,2,3,4,5. Otherwise I might as well continue with Erlang and perhaps Elixir in the future.

Having supervision trees and proper isolation allows you to think about the Error Kernel of your application. In the words of jlouis:

The Error Kernel

Erlang programs have a concept called the error kernel. The kernel is the part of the program which must be correct for its correct operation. Good Erlang design begins with identifying the error kernel of the system: What part must not fail or it will bring down the whole system? Once you have the kernel identified, you seek to make it minimal. Whenever the kernel is about to do an operation which is dangerous and might crash, you "outsource" that computation to another process, a dumb slave worker. If he crashes and is killed, nothing really bad has happened - since the kernel keeps going.

Identifying the kernel plugs the "turtles all the way down" hole. As soon as the kernel is hit, we assume correctness. But since the kernel is small, the trusted computing base of our program is likewise. We only need to trust a small part of the program, and that part is also fairly simple.

A visualization is this: A program is a patchwork of small squares. Some of the squares are red, and these are the "error kernel". Most (naively implemented) imperative programs are mostly red, save for a few squares. These are the squares where exceptions are handled explicitly and the error is correctly mitigated. The kernel is thus fairly large. In contrast, robustness-aware Erlang programs have few red squares - most of the patchwork is white. It is a design-goal to get as few red squares as possible. It is achieved by delegating dangerous work to the white areas so a crash does not affect the kernel.

[Cloven]

Given the possibility of any actor X to supervise any other actor(s) Y, any topology naturally falls out, including trees. So I wouldn't worry too hard about erlang-style supervision trees not being supported; in fact it'd be hard not to support them.

[SeanTAllen]

This is currently blocked by distributed pony work

[SeanTAllen]

Closing this as work is underway and we are soon going to be moving tickets like this over to our RFC system and given that work on distributed Pony is underway, doing an RFC seems like time that could be better spent on other things.