Idea: allow `tell` to yield execution context

[Horusiath]

The idea presented here is to enhance actor-to-actor communication with !/tell by adding to it an optional mechanism for yielding current execution context or letting it to continue execution in recipient actor.

It could be expressed by a return value of tell, which is able to suspend current execution on demand and would be completed, once a message has successfully received (not processed!) by the recipient's mailbox (in case of remote communication, that "recipient" could be an endpoint writer).

//definition
def tell[A](message: A): TellAwaiter

// usage
Behaviors.receive { (ctx, msg) =>
    for {
        // block:1
        _ <- recipient ! Request
        // block:2
    } yield Behaviors.same
}

In C# we have async/await mechanism, which allows us to build an asynchronous state machines. Moreover it allows users to specify both custom state machine builders and awaitable continuations:

//definition
TellAwaiter Tell<T>(T message);

// usage
Behaviors.Receive<T>(async (context, message) => {
    // block:1
    await recipient.Tell(new Request());
    // block:2
    return Behaviors.Same<T>();
});

For motivation I want to cover 2 cases.

1. Passing message to an empty mailbox of idle actor

First case is when an actor A tells message M to another actor B living on the same process. In this case we could reduce an overhead of communication via mailboxes using following approach:

1. A tries to send message M to B.
2. B is idle and has an empty mailbox.
3. _ <- B ! M immediatelly suspends execution on A, and continues processing message M directly on actor B.

In this case we don't even need to put message to the B's mailbox or schedule it via message dispatcher. Instead we could process it immediatelly. This means, that the cost of sending a message between two local actors is virtually zero.

This could be further chained (as B may want to send some message to C etc.). Latency for such cases would be greatly improved. Eventually this would end, causing a suspended continuations to rollback to an original actor, letting it continue it's execution. If such chain is too deep, an Erlang style reductions could be applied - which the difference that reductions work on ! instead of every single function call.

Further idea

With the right approach this could be evaluated further into situation when we could make such message propagation via ask/? as well - in that case we in happy path scenarios we could also reduce ask to be almost an equivalent of function call. However this would probably require that reply is the last statement of actor processing:

Behaviors.receive { (ctx, request: IRepliable[Response]) =>
    for {
        // block:1
    } yield Behaviors.reply(request, Response) then Behaviors.same
}

2. Passing message to actor with full mailbox

On the other side, if we use actors with bounded mailboxes and we want to pass message to an actor with full mailbox, we can use tell continuation to suspend current execution in order to avoid blocking, and to apply backpressure. Example:

1. A tries to send message M to B.
2. B has full mailbox and cannot accept any more messages.
   1. If B was idle, _ <- B ! M immediatelly suspends execution on A, and continues processing on B in order to free its mailbox first.
   2. If B was not idle, we can suspend A anyway and give its quant of processor to another actor that can continue its processing in safe manner.

A similar approach is used in Pony language to apply backpressure to actors.

[ktoso]

Yes, I think it's a pretty interesting idea! "but..." we can't do that in Scala without macros, and we do not want macros in core Akka. So the proposal is very intriguing but I don't think it is actionable.

It would also not be possible to do for Java, which is usually a requirement for all Akka features.

[ktoso]

On the other side, if we use actors with bounded mailboxes and we want to pass message to an actor with full mailbox, we can use tell continuation to suspend current execution in order to avoid blocking, and to apply backpressure. Example:

Yes I've been actually pondering this conceptual possibility recently... The problem is that then we allow actors to be able to lock up. Though if we made them only suspend if they await ! then perhaps such actors know that they may be at risk of deadlock (cycles in awaits), and could be made to work...

I think this is a very interesting area to consider but likely not in Akka Actors hmm...

[viktorklang]

Hi @Horusiath,

Since Akka embraces distributed computing I don't think this proposal will work out in practice since at-most-once delivery makes it impossible.

The feature will not be virtually-zero cost since there will be the cost of tracking mailbox state and mutual exclusion, cost of creating continuations, cost of managing recursion, cost of clearing/setting/restoring contextual information etc). There's also the problem of serializing execution since an actor wouldn't be able to send messages and have them being processed in parallel.

Cheers, √

[Horusiath]

@viktorklang that's why I wrote "virtually" ;) I think, that empty-and-idle check can be done directly on recipient's status flags with a single CAS operation + comparison. Continuation indeed has its weight (in .NET I imagine this could be elided by using value types for optimistic cases), but this is something that should be measured.

Regarding parallel execution: this can be tuned via dispatchers or potentially can even depend on what you want to do with tell's result - if it's lazily evaluated:

Behaviors.receive { (ctx, msg) =>
    for {
        val awaiters = recipients.map(_ ! Request)
        // bellow free current thread until all awaiters complete 
        // (they can be processed in parallel without execution context propagation)
        _ <- whenAll awaiters  
    } yield Behaviors.same
}

This is a common pattern for parallelism in .NET Task - tbh. I'm not sure how applicable it is to something that is supposed to be even more lightweight than Future, just exploring the idea.

[ktoso]

That one is oversimplified though (dangerously misleading if it existed!). Esp // free current thread until all awaiters complete – "complete" means these should be asks, not the awaiting till mailbox reached.

[Horusiath]

@ktoso not necessarily - "awaiter completion" doesn't has to mean message processing. I'm still describing it as a process of putting the message on actor's mailbox.

[viktorklang]

@Horusiath I'm not sure I follow. What semantic value does "putting it on the actor's mailbox" have? I'm not really clear on what problem we're trying to solve. Could we perhaps start over by talking about use-cases?

[Horusiath]

Sorry, maybe I'm interchanging between too many cases.

What semantic value does "putting it on the actor's mailbox" have?

This is related to the situation when we have actors with full, bounded mailbox. Currently in that situation we can do one of two things:

1. Drop the message.
2. Block current thread until the mailbox unlocks.

With awaitable approach, we can also stop executing current actor's code (just leave awaiter so we can return to it later) without dropping messages (they would stay inside awaiter, until mailbox will be emptied) or thread blocking. Additionally if receiver was not processing any message atm. we could continue execution on its side, therefore dequeueing full mailbox.

[He-Pin]

I was thinking about something like this, can Akka suspending the current Actor, and only continues processing after the pipe self invoked. I mean no Stash, new messages will no be scheduled before the the pipe self message return .

processAllSystemMessages() //First, deal with any system messages
processCurrentContinuation()// introduce something like that?
processMailbox() //Then deal with messages

If the actor can yield and will not process any further messages inside the mailbox, then we can implement some more interesting things.