Implementation of ActorSystem, Dispatchers and Executors

[chris-zen]

Hi, recently I have been learning about Scala and Akka and I became amazed by such technologies. The problem is that I have some ongoing projects that are already binded to Python language. I was looking for similar things in Python and found Pykka. It looks great to me, as I don't need all the features that Akka have, so I am considering seriously to start using it. Congratulations for the current work !

However there is something that I don't understand from the current design. Why actors are binded to a given threading technology and can not be decided on runtime, for example from the user configuration ? When you define class MyActor(ThreadingActor) there is no way to use this same actor logic with gevent without touching the source code (let's assume that gevent monkey patch doesn't exist). Or imagine you want to expand pykka to use asyncio and want to start using it in current projects...

From what I have been seeing in the source code it does not look so difficult to solve this problem, but it would require to change a bit the overall design of Pykka. I have already seen the issue #18 and I completely agree with it. This would allow not only having independent actor systems but also having an inbox, future and event loop factory per system, which is the only thing that {Threading,Gevent,Eventlet}Actor's are doing (please correct me if I am wrong).

It would be called ActorSystem and have an actor factory method actorOf (just like in Akka) that would create actor instances with a reference to the actor system that created it. The ActorSystem would implement the methods _create_inbox, _create_future and _create_event_loop that would delegate to the selected threading technology implementation.

ActorSystem would have a constructor that would allow to configure it. As a replacement for Akka Props it could be used the builtin library functools.partial.

Just as a reference:

import pykka
from functools import partial

class Greeter(pykka.Actor):
    def __init__(self, greeting='Hi there!'):
        super(Greeter, self).__init__()
        self.greeting = greeting

    def on_receive(self, message):
        print(self.greeting)

system = pykka.ActorSystem(using=pykka.THREADS)
actor_ref = system.actorOf(partial(Greeter, greeting='Hi you!'))

What do you think ? If we agree on a design I could make a fork and start working on it if anyone else is interested in doing it.

[jodal]

Hi Christian! Thanks for you well researched ideas and good example.

I'm open to changing the model from subclassing specific actor types to having executors live a bit more on the side of the actors themselves. That could also allow for actor systems that have N actors sharing a pool of M executors, where N > M.

I'll keep this issue open and I'll refer to it when I get around to sitting down and working on the next minor or major version of Pykka.

[chris-zen]

Hi @jodal, I see that this is something you have been already thinking about. I have been looking in deep at Akka source code and documentation and there are some important abstractions that could be incorporated to pykka:

• The executor service: The responsible to manage threading and the execution of message handlers. This is the part that should care about Threads, Greenlets, Eventlets, asyncio Coroutines, Queue and Future implementations and so on and so far.
• The dispatcher: responsible for dispatching messages to the executors. There could be different type of dispatchers: 1 actor to 1 executor (each actor have its own thread), N actors to M executors (using a thread pool), N actors to 1 executor (shared executor among a number of related actors).
• The actor system: A system of actors that manages configuration, actor creation, dispatchers, ...

These would be the basic and most important things to work on in pykka. But just to not forget about, there are two other things that could also be incorporated:

• An scheduler: To schedule periodic tasks over the time. (related to #24)
• Event streams: Publish/subscribe system to send/receive events (for example logging, or system statistics, hooks, ...). (related to #16)

If I get some time (...) I will start to work a bit on these things on my fork. Let's see.

[drozzy]

This would indeed be the most-greatest-awesome addition. I would also move the scheduler up there into the important list.