Using monkeypatch in non function scoped fixture

[pytestbot]

Originally reported by: BitBucket: ColeVsCode, GitHub: ColeVsCode

---

Hi, I'd like to be able to create a module or session scoped fixture that uses monkeypatch.

#!python

    @pytest.fixture(scope='session')
    def usermanager(monkeypatch):
    ...

This raises an error. There's some comments about this on the pytest-dev list.

https://mail.python.org/pipermail/pytest-dev/2012-November/002157.html

Is there a work-around in the mean time that I can use to change the scope of the monkeypatch fixture?

---

• Bitbucket: https://bitbucket.org/pytest-dev/pytest/issue/363

[pytestbot]

Original comment by holger krekel (BitBucket: hpk42, GitHub: hpk42):

---

depends on what you wants to do exactly but here is a workaround using internal objects:

#!python

import pytest
from _pytest.monkeypatch import monkeypatch

@pytest.fixture(scope="session")
def monkeysession(request):
    mp = monkeypatch()
    request.addfinalizer(mp.undo)
    return mp

def test_some(monkeysession):
    monkeysession.setattr("os.getcwd", lambda: "/")

def test_other():
    import os
    assert os.getcwd() == "/"

[agamdua]

I would like to monkeypatch the datetime module while running a function and scope it to a module.

I am getting the following issue when I attempt to do so:

ScopeMismatch: You tried to access the 'function' scoped fixture 'monkeypatch' with a 'module' scoped request object, involved factories

with this code (just switched the names around):

@pytest.fixture
def fixture_data(some_data, monkeypatch):
    class MyDate(object):
        @classmethod
        def now(cls):
            return 'date'

    monkeypatch.setattr(
        field_values,
        'datetime',
        MyDate,
    )
    return func_to_test(some_data)

An equivalent with python's unittest seems to be possible pretty easy, I banged this out pretty quick :

import mock
import unittest

import module_to_test

class MyDate(object):
    @classmethod
    def now(cls):
        return 'gibberish'

class MyTestCase(unittest.TestCase):
    @classmethod
    @mock.patch.object(module_to_test, 'datetime', MyDate)
    def setUpClass(self):
        self.test_data = func_to_test(dummy_data)

    def test_func_to_test(self):
        self.assertEqual(self.test_data['create_date'], 'gibberish')

While this "works", I am not able to make this run through the module. I had hoped that it was my inexperience with the library, till I found this issue. Is there anything I can do to help with this? Or is it actually my inexperience with the library? :)

I can point to actual code if desired, I didn't want to swamp maintainers/triagers with too much info.

[nicoddemus]

Hi @agamdua,

Fixtures can only use fixtures of same scope or broader (for example, function-scoped fixture can use session-scoped fixtures, but not the other way around). There was talk about about adding an "any" scope, meaning that a fixture could be used in any scope, but no one's working on it as far as I know.

Did you notice @hpk42 response just before your question? It contains an appropriate workaround to what you are trying to accomplish.

If you prefer to use the mock package, this would be another solution:

@pytest.yield_fixture
def fixture_data(some_data):
    class MyDate(object):
        @classmethod
        def now(cls):
            return 'date'

    with mock.patch.object(field_values, 'datetime', new=MyDate()):
        yield func_to_test(some_data)

Also, if you like the mock package, checkout pytest-mock.

[agamdua]

Hey! Thanks for the response.

Did you notice @hpk42 response just before your question? It contains an appropriate workaround to what you are trying to accomplish.

I did notice the other response - I would have ideally liked to abstract the _pytest import from the rest of the codebase, and I think I can still manage to do that with the method there, something I missed on teh first read.

Thanks also for the example with mock, and pytest-mock packages. The first I didn't think of, and the second I didn't know about.

There was talk about about adding an "any" scope, meaning that a fixture could be used in any scope, but no one's working on it as far as I know.

I will look go through #797 as well, and see if there's anything that comes to mind.

[kunalbhagawati]

The solution above isn't working with pytest==3.0.7 anymore.

Instead I have changed it to

@pytest.fixture(scope="session")
def monkeysession(request):
    mpatch = MonkeyPatch()
    yield mpatch
    mpatch.undo()

Am i doing this right?

[RonnyPfannschmidt]

this is indeed currently necessary, we had to roll back the feature due to a harsh regression

[bossjones]

Is it possible to add @kunalbhagawati's code block to a Common problems section in the documentation at least until the regression is fixed? @RonnyPfannschmidt I think I got bit by this on a test or two. Thanks in advance!

[nicoddemus]

@bossjones that's a good idea. Would you like to contribute a PR? 😉

[bossjones]

Good call, i'll do that :) @nicoddemus

[lukewrites]

Is kunalbhagawati's solution still working, or is there now another workaround? I'm having trouble implementing it in my project – mpatch.undo() throws an error.

[nicoddemus]

@lukewrites it should still be working, nothing has changed in monkeypatch since then. Can you post the full traceback please?

[jaraco]

Just a note, to use kunal's solution, you need from _pytest.monkeypatch import MonkeyPatch:

@pytest.fixture(scope="session")
def monkeysession(request):
    from _pytest.monkeypatch import MonkeyPatch
    mpatch = MonkeyPatch()
    yield mpatch
    mpatch.undo()

[leycec]

@nicoddemus: Would adding the monkeysession fixture revised by @jaraco to the codebase be feasible, hopefully renamed to monkeypatch_session or some such for orthogonality? This fixture is sufficiently useful that it's omission from the pytest package pains my open-sourced soul. Also, the necessity of referencing private attributes (notably, the _pytest.monkeypatch.MonkeyPatch class) renders this solution inherently fragile and liable to explode in everyone's faces with any new pytest release.

I'd help out with a PR to that effect, but... well, I'm lazy. (Also, overworked and underpaid.)

[blueyed]

:+1: for shipping monkeypatch_session as a workaround - often missed it myself.

[kevlarr]

Why do the pytest docs say you can use monkeypatch in a session scope?

[jaraco]

Hmm. They also say "Note that the ability to use a monkeypatch fixture from a session-scoped fixture was added in pytest-3.0." This comment indicates that feature was removed. Probably the documentation is stale. Also, should this issue be re-opened if the feature was rolled back?

[jaraco]

Aah, you were looking at the pytest 3.0.1 docs. The latest docs don't mention that feature.

[kevlarr]

Ah apologies, that's what I get for following Google

[nicoddemus]

Hi,

I don't see invocation-scoped fixtures being implemented at all anytime soon, we expect a good refactoring of the fixture implementation until that can happen.

Given that the implementation of monkeypatch_session is trivial, and even if invocation-scoped fixtures ever come to light it would be a trivial implementation, I'm OK with adding it to the core too. 👍

Just thought I would mention: we had the same issue for years with tmpdir, and we solved that by providing a tmpdir_factory session fixture which would produce tmpdirs by calling mkdir on them. I suppose the same could be done for monkeypatch... does it make sense for different session fixtures to have independent monkeypatch instances? I don't know.

I mention this just because it came to mind, I'm OK with monkeypatch_session.

[RonnyPfannschmidt]

monkeypatch_session should come with documentation ensuring it will at some point be reconciled

a monkeypatch factory/root doesnt make sense as of now (per request would be a nice way for monkeypatch (so every fixture that uses it gets a own one matching its scope)

[sjdemartini]

As of pytest 6.2, this is now quite a bit simpler with no need for a private import, thanks to the pytest.MonkeyPatch class and context-manager, which are available as alternatives to the monkeypatch fixture (https://docs.pytest.org/en/6.2.x/reference.html#pytest.MonkeyPatch):

@pytest.fixture(scope="session")
def monkeysession():
    with pytest.MonkeyPatch.context() as mp:
        yield mp

[wxtim]

IMO this ought to be available as a fixture out of the box since I seem to be using it a lot, and this feels like code to be re-used.