Lazy Result Backend object

[gjeusel]

Hello ! I really liked the LazyBroker interface !

I was wondering if you were planing to add LazyBackend objects also ? (As the backend might also rely on the flask app configuration.)

How about:

DRAMATIQ_BACKEND_CLASS = "RedisBackend"
DRAMATIQ_BACKEND_URL = "redis://localhost:6379"

[epandurski]

I am not sure I understand what you mean by LazyBackend objects.

Currently you can use the Broker class (which does not specify a backend) and then specify the backend in the configuration like this:

DRAMATIQ_BROKER_CLASS = 'RedisBroker'
DRAMATIQ_BROKER_URL = "redis://localhost:6379"

This is mentioned in the documentation: https://flask-melodramatiq.readthedocs.io/en/latest/#configuration

Is this the functionality you need?

[gjeusel]

Sorry ! Let me be more specific ! I would like to be able to do something like this:

import dramatiq
from dramatiq.results import Results
from flask_melodramatiq import Broker
from flask_melodramatiq import Backend

# Lazy Broker, will be evaluated at init_app time
broker = Broker()

# Lazy Backend, will be evaluated at init_app time, either converted into RedisBackend, MemcachedBackend or StubBackend, reading the app config to determine it:
backend = Backend()

broker.add_middleware(middleware=Result(backend=backend))

dramatiq.set_broker(broker)

# So that I can decorate my actors with:
@dramatiq.actor(store_results=True)
def yellow():
    return "I'm Yellow Baby, and I'll be stored"

Usage for Production Purpose

from .app_factory import app_maker
from .drama_broker import broker, backend

class Config:
    DEBUG = False

    # Dramatiq
    DRAMATIQ_BROKER_URL = "redis://localhost:6379/0"
    DRAMATIQ_BROKER_CLASS = "RedisBroker"
    DRAMATIQ_BACKEND_URL = "redis://localhost:6379/1"
    DRAMATIQ_BACKEND_CLASS = "RedisBackend"

app = app_maker(Config)
broker.init_app(app)   # -> turning backend into "RedisBackend" and broker into "RedisBroker"

Usage for UnitTesting Purpose

from pkg.app_factory import app_maker
from pkg.drama_broker import broker, backend

class TestConfig:
    DEBUG = True

    # Dramatiq
    DRAMATIQ_BROKER_URL = None
    DRAMATIQ_BROKER_CLASS = "StubBroker"
    DRAMATIQ_BACKEND_URL = None
    DRAMATIQ_BACKEND_CLASS = "StubBackend"

@pytest.fixture
def app():
    app = app_maker(TestConfig)
    broker.init_app(app)   # -> turning backend into "StubBackend" and broker into "StubBroker"
    return app

[epandurski]

I think I understand what you have in mind now, and you are right that this is not currently supported. I have never used the Results middleware, so I am not sure my thoughts will be of much help but here they are anyway:

1. One possible approach is to solve the more general problem of being able to alter the list of used middleware in the Flask configuration. Currently this is not possible, but I can see a great value in it. Also, it is possible that this will solve your use-case as well.

2. The other approach is what you propose, which should be doable, needs some research about how Dramatiq implements Results and its backends. It seems to me that this will end up to be a quite ugly solution (as is the LazyBroker). I would prefer the first one, if it is doable.

Currently, I do not plan to work on this functionality, but if you come up with a good solution I will be happy to merge it back.