Documentation for running pytest with a different MAIL_BACKEND when using Celery and the app factory pattern

[nickjj]

Hi,

When running tests it's a great idea to use the locmem back-end to avoid really sending out emails through your SMTP provider, but if you happen to use Celery and the app factory pattern it's not straight forward on how to accomplish this.

Normally you'd have a pytest fixture like this:

@pytest.fixture(scope="session")
def app():
    params = {
        "DEBUG": False,
        "TESTING": True,
        "WTF_CSRF_ENABLED": False
    }

    _app = create_app(settings_override=params)

    ctx = _app.app_context()
    ctx.push()

    yield _app

    ctx.pop()

And if you weren't using Celery you could drop a MAIL_BACKEND = "locmem" in the params and you're good to go.

But if you're using Celery this changes because when you create your Celery app as part of starting your Flask app, it won't be running with TESTING = True, which means it never gets set and suddenly if you have a test that hits a Flask URL that sends an email (reset password, etc.) then the above app fixture never gets used.

Having to set the MAIL_BACKEND in development and restart everything just to run your tests with a different backend doesn't seem like a fun workaround either because you have to remember to keep changing this and often times in dev you want a different backend than test.

How would you solve this problem?

[rehmanis]

@nickjj I am not very knowledgeable but can't you override the MAIL_BACKEND using the get_connection() function in the test ?

[nickjj]

Based on one of your tests using that here: https://github.com/waynerv/flask-mailman/blob/1f6ff7e4771062c7364b016ac01909f9f5eacb56/tests/test_backend.py#L95-L104 I'm not sure how that could be applied to my tests because these aren't unit tests on sending the mail out.

For example, imagine this flow:

• You set mail = Mail() in your Flask app factory and init the extension like usual
• You have a Flask view with a User.send_reset_password("foo@example.com") function call
• In your user model you call current_celery_app.send_task("myapp.user.tasks.deliver_reset_password", ["foo@example.com"])
• In your task file for that function you call mail.send(...) where you send the email with Flask-Mailman

And now you have a test which makes a POST request to reset your password. At this point the mailing is done inside of that task file, not the view so there's no way to override get_connection() in the test.

I also wouldn't want to mock out the task function in the test because I'd like to assert the email I'm sending out has the correct information (correct recipient, subject, template text, etc.).

[rehmanis]

In that case pardon me for bringing that up. I haven't used Celery ( I just used plain threading module with flask-mailman) so was not aware of this. Might still not be useful suggestion or a bit hacky but can you not use a fixture to yield a new MAIL_BACKEND and then after yield restore the MAIL_BACKEND to old value ?

[nickjj]

Do you have any suggestions on how to set up that fixture?

[rehmanis]

I think since MAIL config is already initialized you might have to create a different app with the config for email. I wrote some tests to test my configurations for production and development like this https://github.com/Abdur-rahmaanJ/shopyo/blob/dev/shopyo/tests/conftest.py and this https://github.com/Abdur-rahmaanJ/shopyo/blob/dev/shopyo/tests/test_configs.py which might not be what you want. If app.extensions['mailman'].backend = 'locemen' works before yield then that is better.

My email test don't use Celery but in case they are useful: https://github.com/Abdur-rahmaanJ/shopyo/blob/dev/shopyo/modules/box__default/auth/tests/test_email.py

Is it possible to see your pytest that you wrote?

[nickjj]

In the most simple test case, it's:

    def test_home_page(self, app):
        current_celery_app.send_task("myapp.user.tasks.deliver_reset_password", ["foo@examle.com"])

This is running in a class that has the original fixture applied from the issue (the "app" one).

As for the Flask factories themselves outside of the tests, here's example create_app and create_celery_app functions: https://github.com/nickjj/docker-flask-example/blob/505eafcad36c2475d2dd432da665c550d0af33e3/hello/app.py#L14-L60

Edit: I think the issue is the Flask app itself ends up having the correct settings applied due to the fixture, but the Celery app does not and since the email is sent through Celery, those settings have no effect. It ends up running through the create_celery_app that was started when I started the project up. Current issue is I'm not sure how to associate your settings to a custom Celery app but only for tests.

[rehmanis]

I see. So it more of how to setup the celery_app configs and integration. Sorry I was unable to help.

[nickjj]

Yeah, but only during tests because it works fine outside of tests (I can control which backend to use with MAIL_BACKEND).

[waynerv]

I've never used celery, so I'll need some time to investigate the problem you describe.

[caffeinatedMike]

Just came across this package while reviewing alternatives to the no-longer-maintained flask-mail. What attracts me to this package is the ability to create custom backends. Locmem is very useful when developing locally. However, in the event of testing with celery I think it would be worthwhile developing a SQLite backend that stores sent emails. This would allow you to check the emails sent across processes (the main/pytest process and the celery worker process)