celery / jumpstarter

MIT License
7 stars 3 forks source link

Actor Resources #6

Open thedrow opened 3 years ago

thedrow commented 3 years ago

Actors have resources they manage during their lifetime, such as:

A resource is a context manager or an asynchronous context manager. It is entered whenever the Actor is starting, specifically just before the state machine transitions to the starting -> resources_acquired state. It is exited whenever the Actor is stopping, specifically just before the state machine transitions to the starting -> resources_released state (See #3 for details regarding the lifecycle states and transitions).

The @resource decorator

The @resource decorator is the API used to acquire and release the actor's resources. Resources are simply context managers, asynchronous, or synchronous, which are returned from the decorated method.

When resources are acquired, the context manager's __enter__ or __aenter__ method is called. When they are released, the context manager's __exit__ or __aexit__ method is called.

The context managers are entered and exited together concurrently using an AsyncExitStack instance attached to the actor.

Exceptions

Name Description
NotAResourceError Raised when the decorated function returns an object which isn't a context manager or an asynchronous context manager
ResourceAlreadyExistsError Raised when attempting to acquire the same resource more than once
ResourceUnavailable Some resources are optional. Raised when we should skip acquiring the resource
TypeError Raised while acquiring a synchronous resource with a timeout specified

Keyword Arguments

Name Type Description
timeout float or int After the provided amount of time has passed, raise a TimeoutError if the resource is not yet acquired

Synchronous Resources

Synchronous resources are context managers, that is objects with the __enter__ and __exit__ methods defined on them. Synchronous resources do not support timeouts.

They are defined as follows:

from pathlib import Path

from jumpstarter import Actor, resource

class FileHeadActor(Actor):
  def __init__(self, file_path: Path):
    self.file_path = file_path

  @resource
  def log_file(self):
    return open(file_path)

Since synchronous resources are by definition synchronous, we need to acquire & release them using a thread pool. Therefore, we must wrap those objects in a ThreadedContextManager which schedules __enter__ and __exit__ in the thread pool.

Asynchronous Resources

Asynchronous resources are context managers, that is objects with the __aenter__ and __aexit__ methods defined on them. They are defined as follows:

from anyio import open_file
from pathlib import Path

from jumpstarter import Actor, resource

class FileHeadActor(Actor):
  def __init__(self, file_path: Path):
    self.file_path = file_path

  @resource
  async def log_file(self):
    return await open_file(file_path)

They are already implemented in the actor branch.

Internal Resources

Some resources are already defined for each actor. They are acquired and released automatically as they are crucial for the actor's functionality.

There are currently only three internal resources for each actor: an actor state store, a cancel scope and a task group. Both the cancel scope and the task group are essential for implementing #7. The actor state store is essential for implementing #25. Other such resources may be defined in future versions.

Actor State Store

The actor state store is an object for persisting state of actors. It is useful for persisting data that should survive a process shutdown or a crash. See #25 for more information.

Task Group

A task group is an asynchronous context manager that allows you to schedule coroutines as background tasks.

All tasks must finish before the task group is released. By default, a new task group is created if it is not passed as an argument to the start() trigger.

Cancel Scope

A cancel scope is an asynchronous context manager that allows you to cancel coroutines scheduled in the innermost task group that proceeds it.

We use the cancel scope's cancel() method to force the shutdown of the actor.