pathlib.Path.open is not patched when used with create_file and pytest

[ian-h-chamberlain]

After declaring a pathlib.Path at module scope, then the pytest fs fixture to create_file for that path, subsequent calls to pathlib.Path.open fail with "No such file or directory". A quick look at pathlib.Path.open shows that it uses io.open internally.

I'm not sure if there's a straightforward way to patch methods of an object declared at module scope, so maybe this is more of a technical limitation than a bug, but I figured it was worth reporting anyway.

• OS: reproduced on both macOS 10.13.6 and CentOS Linux release 7.3.1611
• Python version: 3.6.4
• Pytest version: 3.4.0
• Pyfakefs version: 3.4.3, also tested against master

Minimal example

# Minimal example of pathlib patching with pyfakefs

import pathlib

EXAMPLE_FILE = pathlib.Path('/test') / 'file'

def test_example_file(fs):
    # NOTE: moving the EXAMPLE_FILE assignment here causes the test to pass

    fs.create_file(EXAMPLE_FILE, contents='stuff here')

    with open(EXAMPLE_FILE) as file:
        assert file.read() == 'stuff here' # this works

    with EXAMPLE_FILE.open() as file:
        assert file.read() == 'stuff here' # this fails

    # other Path methods seem to use Path.open under the hood, so they fail too
    assert EXAMPLE_FILE.read_text() == 'stuff here' # fails
    assert EXAMPLE_FILE.is_file() # fails

Output:

$ pytest test_pathlib.py
======================================== test session starts ========================================
platform darwin -- Python 3.6.4, pytest-3.4.0, py-1.5.2, pluggy-0.6.0
rootdir: /Users/ichamberlain/Downloads, inifile:
plugins: timeout-1.2.0, profiling-1.2.11, pep8-1.0.6, mock-1.6.3, cov-2.5.1, pyfakefs-3.5
collected 1 item

test_pathlib.py F                                                                             [100%]

============================================= FAILURES ==============================================
_________________________________________ test_example_file _________________________________________

fs = <pyfakefs.fake_filesystem.FakeFilesystem object at 0x10ff8eeb8>

    def test_example_file(fs):
        fs.create_file(EXAMPLE_FILE, contents='stuff here')

        with open(EXAMPLE_FILE) as file:
            assert file.read() == 'stuff here' # this works

>       with EXAMPLE_FILE.open() as file:

Users/ichamberlain/Downloads/test_pathlib.py:16:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Users/ichamberlain/.pyenv/versions/3.6.4/lib/python3.6/pathlib.py:1161: in open
    ???
Users/ichamberlain/.pyenv/versions/3.6.4/lib/python3.6/pathlib.py:1015: in _opener
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

pathobj = PosixPath('/test/file'), args = (16777216, 438)

>   ???
E   FileNotFoundError: [Errno 2] No such file or directory: '/test/file'

Users/ichamberlain/.pyenv/versions/3.6.4/lib/python3.6/pathlib.py:387: FileNotFoundError
===================================== 1 failed in 0.36 seconds ======================================

[mrbean-bremen]

Thanks for the report! This is indeed a technical limitation - EXAMPLE_FILE is initialized with the real pathlib and stays that way in the test. One possibility to work around this is to add the module where EXAMPLE_FILE is defined to modules_to_reload, but this is not directly possible if using the fs fixture (see the note in the link).

[mrbean-bremen]

@ian-h-chamberlain - I added an example that shows how to work around your specific problem using a customized fixture.

[ian-h-chamberlain]

@mrbean-bremen thanks for the quick response! In my case I was able to work around it by instantiating another pathlib.Path in the code under test, e.g.

fs.create_file(EXAMPLE_FILE, contents='stuff here')

with pathlib.Path(EXAMPLE_FILE).open() as file:
    assert file.read() == 'stuff here' # this succeeds now

In the future I may decide to use a fixture as in your example. This issue can probably be closed now.

[mrbean-bremen]

Glad that you found a way to get it working!