Recursion Crashes Under Windows in Python 3.12 and 3.13

[mrbean-bremen]

Discussed in https://github.com/pytest-dev/pyfakefs/discussions/1095

^{Originally posted by **MikeTheHammer** December 3, 2024} I think this is a bug, but it might be something I'm doing wrong. I've run the following smoke test under Linux (Ubuntu 24.04) and Windows 10 LTSC, using Pythons 3.9 - 3.13 . ```python def test__pyfakefs_smoke(fs): filename = r"C:\source_file" fs.create_file(filename) with open(filename) as source_file: assert True ``` Under Linux, this test passes on all Pythons 3.9 - 3.13. Under Windows, it passes on 3.9 - 3.11, but crashes with a `RecursionError: maximum recursion depth exceeded` on Python 3.12 and 3.13. On Python 3.12, the stack trace is: ```python ============================= test session starts ============================= platform win32 -- Python 3.12.4, pytest-8.3.4, pluggy-1.5.0 rootdir: C:\Users\Hammer\pyfakefs_smoke plugins: pyfakefs-5.7.2 collected 1 item test_pyfakefs_smoke.py F [100%] ================================== FAILURES =================================== ____________________________ test__pyfakefs_smoke _____________________________ filename = 'C:\\Users\\Hammer\\pyfakefs_smoke\\venv312\\Scripts\\pytest.exe\\__main__.py' module_globals = None def updatecache(filename, module_globals=None): """Update a cache entry and return its list of lines. If something's wrong, print a message, discard the cache entry, and return an empty list.""" if filename in cache: if len(cache[filename]) != 1: cache.pop(filename, None) if not filename or (filename.startswith('<') and filename.endswith('>')): return [] fullname = filename try: > stat = os.stat(fullname) E FileNotFoundError: [WinError 3] The system cannot find the path specified: 'C:\\Users\\Hammer\\pyfakefs_smoke\\venv312\\Scripts\\pytest.exe\\__main__.py' C:\Program Files\Python312\Lib\linecache.py:93: FileNotFoundError ``` That error repeats 74 times, followed by: ```python During handling of the above exception, another exception occurred: fs = def test__pyfakefs_smoke(fs): filename = r"C:\source_file" fs.create_file(filename) > with open(filename) as source_file: C:\Users\Hammer\pyfakefs_smoke\test_pyfakefs_smoke.py:4: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ C:\Users\Hammer\pyfakefs_smoke\venv312\Lib\site-packages\pyfakefs\fake_io.py:93: in open return fake_open( C:\Users\Hammer\pyfakefs_smoke\venv312\Lib\site-packages\pyfakefs\fake_open.py:89: in fake_open if is_called_from_skipped_module( C:\Users\Hammer\pyfakefs_smoke\venv312\Lib\site-packages\pyfakefs\helpers.py:494: in is_called_from_skipped_module stack = traceback.extract_stack() C:\Program Files\Python312\Lib\traceback.py:232: in extract_stack stack = StackSummary.extract(walk_stack(f), limit=limit) C:\Program Files\Python312\Lib\traceback.py:395: in extract return klass._extract_from_extended_frame_gen( C:\Program Files\Python312\Lib\traceback.py:438: in _extract_from_extended_frame_gen f.line C:\Program Files\Python312\Lib\traceback.py:323: in line self._line = linecache.getline(self.filename, self.lineno) C:\Program Files\Python312\Lib\linecache.py:30: in getline lines = getlines(filename, module_globals) C:\Program Files\Python312\Lib\linecache.py:46: in getlines return updatecache(filename, module_globals) C:\Program Files\Python312\Lib\linecache.py:101: in updatecache data = cache[filename][0]() :196: in get_source ??? :534: in _get_data ??? C:\Users\Hammer\pyfakefs_smoke\venv312\Lib\site-packages\pyfakefs\fake_io.py:124: in open_code return self._io_module.open_code(path) C:\Users\Hammer\pyfakefs_smoke\venv312\Lib\site-packages\pyfakefs\fake_io.py:93: in open return fake_open( C:\Users\Hammer\pyfakefs_smoke\venv312\Lib\site-packages\pyfakefs\fake_open.py:89: in fake_open if is_called_from_skipped_module( C:\Users\Hammer\pyfakefs_smoke\venv312\Lib\site-packages\pyfakefs\helpers.py:494: in is_called_from_skipped_module stack = traceback.extract_stack() C:\Program Files\Python312\Lib\traceback.py:232: in extract_stack stack = StackSummary.extract(walk_stack(f), limit=limit) C:\Program Files\Python312\Lib\traceback.py:395: in extract return klass._extract_from_extended_frame_gen( C:\Program Files\Python312\Lib\traceback.py:438: in _extract_from_extended_frame_gen f.line C:\Program Files\Python312\Lib\traceback.py:323: in line self._line = linecache.getline(self.filename, self.lineno) E RecursionError: maximum recursion depth exceeded !!! Recursion detected (same locals & position) =========================== short test summary info =========================== FAILED test_pyfakefs_smoke.py::test__pyfakefs_smoke - RecursionError: maximum... ============================== 1 failed in 0.64s ============================== ``` The stack trace in 3.13 is similar: ```python ============================= test session starts ============================= platform win32 -- Python 3.13.0, pytest-8.3.4, pluggy-1.5.0 rootdir: C:\Users\Hammer\pyfakefs_smoke plugins: pyfakefs-5.7.2 collected 1 item test_pyfakefs_smoke.py F [100%] ================================== FAILURES =================================== ____________________________ test__pyfakefs_smoke _____________________________ filename = 'C:\\Users\\Hammer\\pyfakefs_smoke\\venv313\\Scripts\\pytest.exe\\__main__.py' module_globals = None def updatecache(filename, module_globals=None): """Update a cache entry and return its list of lines. If something's wrong, print a message, discard the cache entry, and return an empty list.""" # These imports are not at top level because linecache is in the critical # path of the interpreter startup and importing os and sys take a lot of time # and slows down the startup sequence. import os import sys import tokenize if filename in cache: if len(cache[filename]) != 1: cache.pop(filename, None) if not filename or (filename.startswith('<') and filename.endswith('>')): return [] fullname = filename try: > stat = os.stat(fullname) C:\Program Files\Python313\Lib\linecache.py:100: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ('C:\\Users\\Hammer\\pyfakefs_smoke\\venv313\\Scripts\\pytest.exe\\__main__.py',) kwargs = {}, should_use_original = True @functools.wraps(f) def wrapped(*args, **kwargs): should_use_original = FakeOsModule.use_original if not should_use_original and args: self = args[0] fs: FakeFilesystem = self.filesystem if self.filesystem.patcher: skip_names = fs.patcher.skip_names if is_called_from_skipped_module( skip_names=skip_names, case_sensitive=fs.is_case_sensitive, ): should_use_original = True if should_use_original: # remove the `self` argument for FakeOsModule methods if args and isinstance(args[0], FakeOsModule): args = args[1:] > return getattr(os, f.__name__)(*args, **kwargs) E FileNotFoundError: [WinError 3] The system cannot find the path specified: 'C:\\Users\\Hammer\\pyfakefs_smoke\\venv313\\Scripts\\pytest.exe\\__main__.py' C:\Users\Hammer\pyfakefs_smoke\venv313\Lib\site-packages\pyfakefs\fake_os.py:1454: FileNotFoundError ``` This is repeated 60 times, followed by: ```python During handling of the above exception, another exception occurred: fs = def test__pyfakefs_smoke(fs): filename = r"C:\source_file" fs.create_file(filename) assert os.path.exists(filename) > with open(filename) as source_file: C:\Users\Hammer\pyfakefs_smoke\test_pyfakefs_smoke.py:7: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ C:\Users\Hammer\pyfakefs_smoke\venv313\Lib\site-packages\pyfakefs\fake_io.py:93: in open return fake_open( C:\Users\Hammer\pyfakefs_smoke\venv313\Lib\site-packages\pyfakefs\fake_open.py:89: in fake_open if is_called_from_skipped_module( C:\Users\Hammer\pyfakefs_smoke\venv313\Lib\site-packages\pyfakefs\helpers.py:494: in is_called_from_skipped_module stack = traceback.extract_stack() C:\Program Files\Python313\Lib\traceback.py:260: in extract_stack stack = StackSummary.extract(walk_stack(f), limit=limit) C:\Program Files\Python313\Lib\traceback.py:445: in extract return klass._extract_from_extended_frame_gen( C:\Program Files\Python313\Lib\traceback.py:492: in _extract_from_extended_frame_gen f.line C:\Program Files\Python313\Lib\traceback.py:369: in line self._set_lines() C:\Program Files\Python313\Lib\traceback.py:350: in _set_lines lines.append(linecache.getline(self.filename, lineno).rstrip()) C:\Program Files\Python313\Lib\linecache.py:25: in getline lines = getlines(filename, module_globals) C:\Program Files\Python313\Lib\linecache.py:41: in getlines return updatecache(filename, module_globals) C:\Users\Hammer\pyfakefs_smoke\venv313\Lib\site-packages\pyfakefs\fake_filesystem_unittest.py:702: in updatecache return self.linecache_updatecache(filename, module_globals) C:\Program Files\Python313\Lib\linecache.py:108: in updatecache data = cache[filename][0]() C:\Program Files\Python313\Lib\linecache.py:189: in get_lines return get_source(name, *args, **kwargs) :196: in get_source ??? :617: in _get_data ??? C:\Users\Hammer\pyfakefs_smoke\venv313\Lib\site-packages\pyfakefs\fake_io.py:124: in open_code return self._io_module.open_code(path) C:\Users\Hammer\pyfakefs_smoke\venv313\Lib\site-packages\pyfakefs\fake_io.py:93: in open return fake_open( C:\Users\Hammer\pyfakefs_smoke\venv313\Lib\site-packages\pyfakefs\fake_open.py:89: in fake_open if is_called_from_skipped_module( C:\Users\Hammer\pyfakefs_smoke\venv313\Lib\site-packages\pyfakefs\helpers.py:494: in is_called_from_skipped_module stack = traceback.extract_stack() C:\Program Files\Python313\Lib\traceback.py:260: in extract_stack stack = StackSummary.extract(walk_stack(f), limit=limit) C:\Program Files\Python313\Lib\traceback.py:445: in extract return klass._extract_from_extended_frame_gen( C:\Program Files\Python313\Lib\traceback.py:492: in _extract_from_extended_frame_gen f.line C:\Program Files\Python313\Lib\traceback.py:369: in line self._set_lines() E RecursionError: maximum recursion depth exceeded !!! Recursion detected (same locals & position) =========================== short test summary info =========================== FAILED test_pyfakefs_smoke.py::test__pyfakefs_smoke - RecursionError: maximum... ============================== 1 failed in 1.20s ============================== ``` Versions: - pytest 8.3.4 - pyfakefs 5.7.2 - Linux Ubuntu 24.04 - Linux 6.8.0-49-generic x86_64: - Python 3.9.19 - Python 3.10.14 - Python 3.11.9 - Python 3.12.4 - Python 3.13.0 - Windows 10 LTSC 21H2 (OS Build 19044.5131) : - Python 3.9.13 - Python 3.10.11 - Python 3.11.9 - Python 3.12.4 - Python 3.13.0

[mrbean-bremen]

I have seen a somewhat similar behavior with #1086, but labeled it as a limitation due to another context. But if that happens with such a basic test, it is certainly a bug, at least as far as I can see now.

I did not really understand the cause of the problem (it has to do with the virtual environment and the Python path, but I was not able to find a good solution to prevent this).

An easy workaround to fix this is to use python -m pytest instead of calling pytest directly. This is generally considered good practice, but the direct call should also work. I cannot promise a timely fix, as I already have been struggling with this without success, but I will see what I can do.

[MikeTheHammer]

Yes, running it as python -m pytest works for me with both 3.12 and 3.13.