Resources for learning about effect handlers

Hi there, Thanks for your interest in jafx! :)

Effect handlers are sometimes also known as extensible effects or algebraic effects. AFAIK they were originally proposed by Oleg Kiselyov et al. in https://dl.acm.org/doi/pdf/10.1145/2503778.2503791, though I must admit that I'm not very familiar with the literature. Essentially, the idea is to make function calls hot-swappable in the sense that they can have different "effects" (state modifications/IO) depending on in which context they are run.

The effect handlers in jafx are quite simple. Below is a basic implementation and example:

from abc import abstractmethod
from dataclasses import dataclass

# Globals to keep track of the effect handlers in the current context
stack = []
stack_idx = []

@dataclass
class Message:
    pass

class Handler:
    def __enter__(self):
        stack.append(self)
        return self

    def __exit__(self, exc_type, exc_value, traceback):
        _ = stack.pop()

    @abstractmethod
    def is_handler_for(self, message: Message) -> bool:
        """Return True if this handler can handle a given message/effect"""
        pass

    @abstractmethod
    def __call__(self, message: Message):
        """Handle a message/effect"""
        pass

def interpret(message: Message, interpret_final: bool = True):
    """Interprets a message/effect"""

    # We either interpret the message using the full (final) handler stack or,
    # when handling a partially interpreted message, the remaining stack. See
    # example below with `ExclamationMarkAdder`.
    if interpret_final:
        cur_stack_idx = len(stack)
    else:
        cur_stack_idx = stack_idx[-1]
    stack_idx.append(cur_stack_idx)

    # Loop over handler stack, applying the handlers in reverse order
    for handler in reversed(stack[:cur_stack_idx]):
        stack_idx[-1] -= 1
        if handler.is_handler_for(message):
            return handler(message)
    else:
        raise RuntimeError("No handler for message: {}".format(message))

@dataclass
class PrintMessage(Message):
    message: str

class PrintHandler(Handler):
    """Handler for PrintMessage effect. Prints a message to stdout."""

    def is_handler_for(self, message: Message) -> bool:
        return isinstance(message, PrintMessage)

    def __call__(self, message: PrintMessage):
        print(message.message)

class ExclamationMarkAdder(Handler):
    def is_handler_for(self, message: Message) -> bool:
        return isinstance(message, PrintMessage)

    def __call__(self, message: PrintMessage):
        # Intercepts a PrintMessage and adds an exclamation mark to the
        # messsage. We then interpret the modified message with the remaining
        # stack (`interpret_final=False`).
        message.message += "!"
        return interpret(message, interpret_final=False)

def print_eff(s: str):
    """Effectful variant of `print`"""
    interpret(PrintMessage(s))

def main():
    print_eff("Hello, world")

# main()
# ^ Running this gives us:
# RuntimeError: No handler for message: PrintMessage(message='Hello, world')

with PrintHandler():
    main()  # prints "Hello, world"

with PrintHandler(), ExclamationMarkAdder(), ExclamationMarkAdder():
    main()  # prints "Hello, world!!"

Note that the implementation above has several weaknesses. It is, for example, not compatible with async and multithreading. It also has other shortcomings compared to the effect handlers proposed by Kiselyov et al. In their paper, the handler takes a continuation and is thus able to run the remaining computation any number of times (including zero). Unfortunately, Python does not natively support continuations, so implementing proper effect handlers is difficult.

Pyro has a very slimmed down implementation called minipyro: https://pyro.ai/examples/minipyro.html. I found it very instructive when learning about how Pyro and the effect system in Pyro works. Can definitely recommend!

ludvb / jafx

Resources for learning about effect handlers #1