VSCode with Python 3.11.05a+: Warning message when starts

[YvesDup]

When running the debugger of VSCode with Python 3.11.05a, this message prints:

This version of python seems to be incorrectly compiled (internal generated filenames are not absolute). This may make the debugger miss breakpoints. Related bug: http://bugs.python.org/issue1666807

And the related link is a very old issue (more than a decade)

My configuration is:

• Mac OSX M1 pro, 12.0.1
• VSCode, version 1.64.2 (Universal)
• Python 3.11.05a+

[fabioz]

Well, the paths being reported by CPython are wrong in this case (they're relative instead of absolute), so, it's a CPython regression probably...

This is the 2nd report about that (the first one is in debugpy: https://github.com/microsoft/debugpy/issues/847).

There's not much I can do about it, this needs to be fixed in CPython (the debugger already works around it, but it notifies users so that they know there's something wrong in the Python version being used).

[gvanrossum]

@fabioz, CPython core dev here. Can you help us track this down? I tried to run the reproducer script posted in http://bugs.python.org/issue1666807 but I could not trigger the issue, so something else seems to be wrong. At the very least the error message is misleading (since the problem is not the same as in the bpo issue linked).

In 3.11 quite a few internals of code objects (and other aspects of the interpreter) that might be of interest to pydevd have changed, so is it possible that changes to pydev might need to be made?

[fabioz]

Hi @gvanrossum hehe, quite a core dev ;)

I must say I also don't really know what happens, all I know is that doing:

import os.path
print(os.path.realpath.__code__.co_filename)

Is returning a relative path and not an absolute path as it should.

I agree the error message is misleading (this check was done back when that bug was rampant, but that bug is probably not applicable anymore and does raise confusion). I'll fix that in the debugger (reopening to improve the error message).

[gvanrossum]

Ah, that particular module is "frozen" (which is a hack to speed up startup), and its filename shows as <frozen ntpath>. That's meant to be a feature, and it only happens for frozen modules.

You can disable frozen modules (except for some bootstrap code in importlib) by passing -Xfrozen_modules=off on the Python commandline.

[fabioz]

Humm, will standard modules will be frozen by default? (i.e.: is it opt in or opt out?)

I guess that the debugger needs to pass -Xfrozen_modules=off by default with Python 3.11 (because it's not possible for the debugger to debug frozen modules as the filename is definitely required to hit breakpoints).

[gvanrossum]

Humm, will standard modules will be frozen by default? (i.e.: is it opt in or opt out?)

Only a few modules that are always loaded at startup (the list is here: https://github.com/python/cpython/blob/main/Python/frozen.c, generated here: https://github.com/python/cpython/blob/main/Tools/scripts/freeze_modules.py), though there are requests for a larger set. Definitely not the whole stdlib though.

I guess that the debugger needs to pass -Xfrozen_modules=off by default with Python 3.11 (because it's not possible for the debugger to debug frozen modules as the filename is definitely required to hit breakpoints).

That would make sense. We could try to patch the filename on each individual code object but that would waste time and memory we're trying to save at startup.

I wonder if you could parse the string <frozen XXX> and use XXX to look up the module in sys.modules? The module object's __file__ attribute is set correctly on startup for frozen modules.

(Also, I believe this is actually specific to deep-frozen modules. "Classic" frozen modules go through marshal which patches co_filename. But they're going out of style except for bootstrapping.)

[fabioz]

I wonder if you could parse the string <frozen XXX> and use XXX to look up the module in sys.modules? The module object's __file__ attribute is set correctly on startup for frozen modules.

That seems like a reasonable workaround in this case if the module file points to the proper file even though the code object doesn't.

[gvanrossum]

FWIW a better description of the older issue is here: https://bugs.python.org/issue1180193. At the time the problem happened when the PYC file was generated in one location but then PY and PYC files were moved elsewhere. The patch there updates the co_filename attributes in-memory after they have been read. It was applied to 2.6+ and 3.0+ so the problem was essentially solved -- until deep-freezing caused it to pop up again. I would recommend passing the -X flag over parsing the <frozen XXX> string, because when you're debugging, well, who cares about the extra msec saved at startup. :-)

[fabioz]

FWIW a better description of the older issue is here: https://bugs.python.org/issue1180193.

Thanks, I think I'll just omit the bug reference as those versions aren't even supported anymore.

I would recommend passing the -X flag over parsing the <frozen XXX> string, because when you're debugging, well, who cares about the extra msec saved at startup. :-)

Agreed.

[fabioz]

The error message was improved in: https://github.com/fabioz/PyDev.Debugger/commit/74aa8fab0aa029e16db1078edf2d52298ed482de

Better support for Python 3.11 frozen modules will be done when https://github.com/microsoft/debugpy/issues/861 is tackled.

[gvanrossum]

A workaround is to add this to the configuation in launch.json:

"pythonArgs": ["-Xfrozen_modules=off"],

[YvesDup]

@gvanrossum FYI the error message is correct from version 2022.4.1

0.00s - Debugger warning: It seems that frozen modules are being used, which may 0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off 0.00s - to python to disable frozen modules. 0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.

A PR is submitted (see https://github.com/microsoft/debugpy/pull/937)

[Resonanz]

Python 3.11.0 Windows 10 VSCode 1.73.1

0.01s - Debugger warning: It seems that frozen modules are being used, which may 0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off 0.00s - to python to disable frozen modules. 0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.

This did not fix it:

"pythonArgs": ["-Xfrozen_modules=off"],

Full VSCode launch.json:

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "main.py",
            "type": "python",
            "request": "launch",
            "program": "main.py",
            "console": "integratedTerminal",
            "justMyCode": true,
            "pythonArgs": ["-Xfrozen_modules=off"]  // Added to fix warning: https://github.com/fabioz/PyDev.Debugger/issues/213
        },

        {
            "name": "Python: Current File",
            "type": "python",
            "request": "launch",
            "program": "${file}",
            "console": "integratedTerminal",
            "justMyCode": true,
            "pythonArgs": ["-Xfrozen_modules=off"]  // Added to fix warning: https://github.com/fabioz/PyDev.Debugger/issues/213
        }
    ]
}

Not sure if this is relevant, but the warning disappears if I comment out the multiprocessing call:

comms_process = Process(target=CommsClass.do_stuff, daemon=True, args=('bob',23,34,45,17,)).start()

class CommsClass():
    def do_stuff(*args):
        while True:
            print('hello ', args[0] + " " + str(args[1]))
            time.sleep(0.1);

[MalikRumi]

I don't know why this is 'closed' when it has not been fixed, but I wanted someone to know that I got this error message on PyCharm today, so the error is not limited to vsCode. My os is Ubuntu 22.04 LTS. I downloaded this Python from deadsnakes.

With respect to the workaround, is that something I am supposed to run every time I invoke Python, or is it 'once for all'? Either way, it is not working for me:

`

f1 = inspect.getabsfile(inspect) f2 = inspect.getabsfile(inspect.iscode) print('File for inspect :',f1) File for inspect : /usr/lib/python3.11/inspect.py print('File for inspect.iscode:',f2) File for inspect.iscode: /usr/lib/python3.11/inspect.py print('Do these match?',f1==f2) Do these match? True if f1==f2: ... print('OK') ... else: ... print('BUG - this is a bug in this version of Python') ... OK -Xfrozen_modules=off File "", line 1 -Xfrozen_modules=off ^^^^^^^^^^^^^^^^ SyntaxError: cannot assign to expression here. Maybe you meant '==' instead of '='? -Xfrozen_modules==off Traceback (most recent call last): File "", line 1, in NameError: name 'Xfrozen_modules' is not defined `

Maybe this line goes somewhere else in PyCharm, but I don't know where that place would be.

I assume the point was that the test would not pass, but as you can see, it does for me.

Please advise. Thanks.

[douglatornell]

@MalikRumi This issue has been resolved in PyCharm 2022.3.

In 2022.2 the workaround was to put -Xfrozen_modules=off in the "Interpreter options:" field of the Run/Debug Configurations panel for each of the debug configurations.

Hope that helps...

[lsmith77]

@MalikRumi This issue has been resolved in PyCharm 2022.3.

In 2022.2 the workaround was to put -Xfrozen_modules=off in the "Interpreter options:" field of the Run/Debug Configurations panel for each of the debug configurations.

Hope that helps...

@douglatornell can you spell it out a bit more in detail how to set "Interpreter options" ?

[douglatornell]

I hope this screenshot helps, @lsmith77.

[ctb-ux]

-Xfrozen_modules=off

This hasn't worked for me. Do you have any other suggestions thanks?

[jimothyGator]

When I put "pythonArgs": ["-Xfrozen_modules=off"], in my launch.json, I still get the frozen modules warning, as others have said. Running Python 3.11.1.

Running ps -aux reveals this process:

[...]/.direnv/python-3.11.1/bin/python -Xfrozen_modules=off -X frozen_modules=off [...]/.vscode/extensions/ms-python.python-2023.2.0/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy --connect 127.0.0.1:54677 --configure-qt none --adapter-access-token [...] ./manage.py runserver

What's interesting about this is -Xfrozen_modules=off -X frozen_modules=off. It's repeating frozen_modules=off, the second time with a space after -X. Perhaps that's an issue?

My launch config:

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Django",
            "type": "python",
            "request": "launch",
            "program": "./manage.py",
            "pythonArgs": ["-Xfrozen_modules=off"],
            "console": "internalConsole",
            "args": [
                "runserver",
            ],
            "django": true,
            "justMyCode": true,
        }
    ]
}

[jimothyGator]

This debuggy issue explains the source of the second -X frozen_modules=off. That argument has been added by default, so it should not be necessary to add pythonArgs to the launch config.

It does not, however, explain why I still get the warning.

[TheManchineel]

I still get this warning trying to debug a FastAPI app invoked with Uvicorn:

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python: FastAPI",
            "type": "python",
            "request": "launch",
            "module": "uvicorn",
            "pythonArgs": [
                "-Xfrozen_modules=off"
            ],
            "args": [
                "my_app:app",
                "--reload"
            ],
            "jinja": true,
            "justMyCode": true
        }
    ]
}

I'm on Python 3.11.2. Any advice?

[h12sw06]

Uvicorn으로 호출된 FastAPI 앱을 디버깅하려고 하면 여전히 이 경고가 표시됩니다.

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python: FastAPI",
            "type": "python",
            "request": "launch",
            "module": "uvicorn",
            "pythonArgs": [
                "-Xfrozen_modules=off"
            ],
            "args": [
                "my_app:app",
                "--reload"
            ],
            "jinja": true,
            "justMyCode": true
        }
    ]
}

저는 Python 3.11.2를 사용하고 있습니다. 어떤 충고?

"env": {"PYDEVD_DISABLE_FILE_VALIDATION": "1"}

Try registering together