[SUGGESTION] Metaclasses and Actors

[smallstepforman]

Hi Herb. I greatly respect your work over the years on C++ and on cppfront, we're lucky to have you.

I've become a big fan of the Actor programming model, and introducing it to all my multithreaded projects. From a C++ point of view, an Actor is an object with a std::deque<std::function<void ()> >, and the runtime provides a thread pool (with work stealing). Clients add method pointers with arguments to a queue, that are invoked asynchronously from the target Actors thread. This is great for communicating objects which manage a resource that run on different threads (eg. video decoding, pushing audio buffers to sound card, networking, GPU rendering etc). When using a C++ Actor library, the biggest obstacle is requiring engineers to be disciplined, ie. them must never invoke an Actors methods directly (eg. DoStuff), where in former case the method will be invoked from the calling Actors thread (which is bad), instead the developer must queue message via the Async(object::DoStuff, args) pathway so that the method will be scheduled to run on the target Actors thread. In essence, the Actor methods need to be private to direct invocation, and public to queueing.

cppfront allows developers to create a compiler contract via metaclass. Would it be possible for the user to create an Actor metaclass which restricts direct method access (ie. the methods are private), yet allow queueing the message via a public Async pathway? A async keyword / typedef would help identify such a function.

Also, with operator overloading, it would be ideal if we can overload -> (or similar) so that the developer when reading the following piece of code target->Foo(), they immediately know that the method is queued to run asynchronously.

Since I'm sharing this moment with you, during my transition to the actor programming model, I have learned that there is a requirement for synchronous Actor access as well. The solution in my Actor framework (https://github.com/smallstepforman/Medo/blob/main/Actor/Actor.h) was to introduce a Actor lock, which internally sets a "do not schedule" flag for the actor, and any queued messages are delayed while the Actor is locked. This allows the calling code to synchronously work with the actor and guarantee that no other thread will modify the Actor. Once the actor is unlocked, the scheduler will continue invoking queued messages. In my ideal programming language, the compiler will understand if an Actor is locked and allow direct Actor method access while locked.

Herb, similar to your work on cppfront, I'm doing something similar with these actor concepts (lets call it ActorFront), which I'd gladly abandon if I there was a pathway where I can add these concepts to cppfront. Ideally the language would allow me to somehow express these concepts (without adding an actor built-in type to the core language), and offload the rest to a library. A big ask :)

[hsutter]

Thanks! Quick ack: It sounds like you're thinking along the same lines as what I've called 'active objects.' A very early article I wrote about it bit-rotted out of DDJ but is still available here: https://www.state-machine.com/doc/Sutter2010a.pdf

Yes, my goal is to generate this using type metafunctions, that (very roughly) convert code like this

background_worker: @active type  = 
{
    save: ( filename: std::string ) -> int = { … }

    print: ( inout data: Data ) = { … }

    private_data: SomePrivateStateAcrossCalls = ();
}

to something like this, where active_helper wraps a thread and a message queue as mentioned in the above article:

background_worker: type  = 
{
    save: ( filename: std::string ) -> std::future<int> =  {
        p: std::promise<int> = ();
        f := p.get_future();
        a.send( :() = p$.set_value( do_save(filename$) ) );
    }

    print: ( inout data: Data ) = a.send( :() = do_print(data&$*) );

    private do_save: ( copy filename: std::string ) -> int = { … }

    private do_print: ( inout data: Data ) = { … }

    private_data: SomePrivateStateAcrossCalls = ();  // only accessed from a's private worker thread
    a: active_helper;
}

I plan to make this work, though it doesn't yet as I still need to add reflection for function parameters. But I think it sounds a bit like what you describe?

[smallstepforman]

Hi Herb. Thank you for taking the time to look at my suggestion. I'm really excited about cppfront, and hope to one day write all new code with it.

Your article about active object (in my opinion) introduces readers to the initial actor like design pattern. A "full" actor runtime has a shared thread pool initially equal to the CPU core count (so that you can have a million actors and not exhaust the OS thread list), and the runtime monitors if the system is "busy" before adding a new thread to the pool (with an upper limit, say 3x cpu core count). The actor runtime also does work stealing, so actors can migrate from their initially assigned thread if that thread is busy. This is a great feature of the design, when you queue a message, the runtime will check if that thread is busy and organise migration. Likewise, when a thread is idle, it will steal work. It's a great load balancer for modern systems, especially when you read about how CPU designers now days halt a hot CPU core for a bit so that it can cool down :) Scary.

I was never an advocate for cpp11 futures and promises, since in the backend, all it does in spawn a thread and a create a condition variable, and caller blocks on the wait call, and the future function will signal the condition. I think the abstract is dangerous since there is so much happening under the covers, and it doesn't scale well since every future is a new thread+condition variable combo, and the OS is constantly creating/tearing down threads. One could do this manually, with almost the same number of lines of code. I dont know how well it scales. The compiler writers probably also have a shared thread pool for futures.

I think Actor messaging is easier to reason about than futures/promises. You mentally know an actor is running on a different thread, you queue messages, and you can receive a message when its done. If you want to mimic a future/promise, the caller creates a condition variable and adds a reference as a message argument, the client actor will signal the condition when it eventually runs, and unlock the caller. But here is another benefit of doing it manually. You may want 10 different actors helping you out. Just have a counting_semaphore. Once all 10 actors signal (increment the count) the counting semaphore, eventually the caller gets unlocked. I just find the actor design pattern a more "generic" patter which covers a lot more use cases than isolated threads, futures/promises, condition variables, etc, I think the abstraction is "purer".

Getting back to cppfront and meta classes, I do like that you are thinking of a @active object metaclass. This is exactly what I was hoping for. Your article tries to abstract the messaging system, by having a pair of functions (eg. Foo() and doFoo(), where Foo() is public and does the queueing, and doFoo() runs in a thread). Given a preference, I would prefer to have a myactor.async(doFoo, args) like function. In my ideal programming langauge, there would be syntatic sugar like myactor->DoFoo(args), where the -> is an overload for the async function (when I think async, it is equivalent to QueueMessage). With UFCS, there is no longer a need for ->, so we might as well repurpose it to do async queueing.

All of the above is just icing sugar, the real challenge is to have the compiler know for @actor (or @active) metaclasses, there will be a set of methods/functions which can only be invoked via the Async mechanism, and never be called directly. The compiler will validate this. Calls to doFoo can only be done via the async mechanism. Your linked article does this trick by having a pair of functions (Foo and doFoo). I guess this also does the intended job (and I have used this pattern for controlling a audio playback manager, which runs in a different thread). As an engineer, I do like async calls to be obvious when reading the code. When I see myactor.Foo() I do not immediately realise that this method will in the background pass an async message. I would prefer to read myactor.Async(doFoo, args) or even better myactor->doFoo(args) and immediately know that this is asynchronous. And hopefully, the compiler will prevent me from manually invoking myactor.doFoo(args); //error, this is invoked synchronously.

At this point in time, I wont mention that engineers being engineers, will also want a backend mechanism to call async functions synchronously. The real world problems destroy pure abstractions. My actor library does this with a sync mechanism, and relies on the engineer to lock the actor first. Locking means setting a "do not schedule" flag, which the actor runtime respects. When the actor is not currently executing a message, then it locks and the caller can run their sync code. And that actor is only ever run from a single thread at a time, respecting the core philosophy of not touching data from 2 threads simultenously.

If you have gotten this far, thank you for spending the team reading about my experiences and thoughts about actors and messaging and code clarity.

[hsutter]

Thanks! I do think we're mostly on the same page:

• Yes, the option of taking a callback rather than returning a future is a valid one. It's especially effective when you pass another Active object method as a callback, since it's inherently an async message callback.
• Yes, I'm happy to introduce "async" or similar notation there. What would you think of all the public functions being named with an async_ prefix?
• Yes the option of running on a thread pool / work stealing runtime instead of spinning up a thread per active object is a more modern and scalable implementation. For teaching I show a thread for familiarity, but try to use the word "conceptually" in there by saying "conceptually each object runs on its own thread." However, letting active objects move across threads has a usability tradeoff, in that we need to teach people not to hold locks across method calls, or otherwise depend on the thread ID, which isn't always visible so I wonder if it would be a long-term pain point.
• You mention some methods being direct (not async messages). Do you mean (a) direct immediate synchronous calls, or (b) async calls where the caller blocks? I've avoided (a) because it seems like a pitfall, including that now the active object's internal state would require synchronization (or there would have to be some way to forbid such a message from accessing it). Instead the option I've taught is a priority queue, so that some messages can have higher priority and not wait behind earlier 'normal' messages, but are still async and must run on the active object's (conceptual) thread. If you mean (b) that's more tractable but, well, it introduces blocking.

A small note: A std::future is not tied to a thread, it's intended to be a general vocabulary type... every std::future and std::promise is just basically a slightly-glorified shared_ptr< struct { T result; atomic_bool ready; > } > (plus a thread-safe get and set API, respectively). and can be used generally as an async result type whether that result is produced by a work stealing pool thread or something else. I don't have data on whether it's been adopted broadly in the field as such though, which would confirm/disconfirm its design and usability.

[jcanizales]

Yes the option of running on a thread pool / work stealing runtime instead of spinning up a thread per active object is a more modern and scalable implementation. For teaching I show a thread for familiarity, but try to use the word "conceptually" in there by saying "conceptually each object runs on its own thread." However, letting active objects move across threads has a usability tradeoff, in that we need to teach people not to hold locks across method calls, or otherwise depend on the thread ID, which isn't always visible so I wonder if it would be a long-term pain point.

This is very similar if not equivalent to how Chrome is written. When I started working on that codebase, I remember this was a very neat explanation of what to do and what not to do, and why (it's shorter than the table of contents suggests).

The Chromium classes end up with a good amount of boilerplate, but as explained in that doc it's for a very good reason. (You can probably find example classes by searching for one of the CHECK macros).

If cpp2's meta capabilities were able to encapsulate all that away into a library, that's a very important and notable set of guidelines that wouldn't need to be taught anymore, the resulting code would more succinctly express intent only, and the programmers wouldn't have to remember to do those rituals right.