microsoft / vscode-python-debugger

Python debugger (debugpy) extension for VS Code.
https://marketplace.visualstudio.com/items?itemName=ms-python.debugpy
MIT License
54 stars 22 forks source link

OSError on reload when running Werkzeug server with vs code debugger #180

Open sergehauri opened 1 year ago

sergehauri commented 1 year ago

Type: Bug

Behaviour

Expected vs. Actual

The Werkzeug development server features a reloader, which checks the files of the wsgi application it's running and restarts the server if there are any changes. In Werkzeug 2.2.3, they fixed an issue with unclosed sockets the development server (issue: https://github.com/pallets/werkzeug/issues/2421, fix: https://github.com/pallets/werkzeug/pull/2517).

With that fix, running the werkzeug server with the vs code debugger causes the following exception when reloading the app:

Exception in thread Thread-6 (serve_forever):
Traceback (most recent call last):
  File "C:\Users\TAAHASE0\AppData\Local\Programs\Python\Python310\lib\threading.py", line 1009, in _bootstrap_inner
    self.run()
  File "C:\Users\TAAHASE0\AppData\Local\Programs\Python\Python310\lib\threading.py", line 946, in run
    self._target(*self._args, **self._kwargs)
  File "c:\Users\TAAHASE0\dev\projects\werkzeugtest\venv\lib\site-packages\werkzeug\serving.py", line 766, in serve_forever
    super().serve_forever(poll_interval=poll_interval)
  File "C:\Users\TAAHASE0\AppData\Local\Programs\Python\Python310\lib\socketserver.py", line 232, in serve_forever
    ready = selector.select(poll_interval)
  File "C:\Users\TAAHASE0\AppData\Local\Programs\Python\Python310\lib\selectors.py", line 324, in select
    r, w, _ = self._select(self._readers, self._writers, [], timeout)
  File "C:\Users\TAAHASE0\AppData\Local\Programs\Python\Python310\lib\selectors.py", line 315, in _select
    r, w, x = select.select(r, w, w, timeout)
OSError: [WinError 10038] An operation was attempted on something that is not a socket

The specific change causing this behaviour is the finally block in src/werkzeug/serving.py on lines 1066-1067.

        try:
            run_with_reloader(
                srv.serve_forever,
                extra_files=extra_files,
                exclude_patterns=exclude_patterns,
                interval=reloader_interval,
                reloader_type=reloader_type,
            )
        finally:
            srv.server_close()

Steps to reproduce:

  1. Use a venv with flask and werkzeug 2.2.3 installed
  2. app.py
    
    from flask import Flask

app = Flask(name)

3. launch.json

{ // Use IntelliSense to learn about possible attributes. // Hover to view descriptions of existing attributes. // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387 "version": "0.2.0", "configurations": [ { "name": "Python: Flask", "type": "python", "request": "launch", "module": "flask", "env": { "FLASK_APP": "app.py", "FLASK_DEBUG": "1" }, "args": [ "run", "--no-debugger" ], "jinja": true, "justMyCode": true } ] }

4. "Start Debugging"
vs code runs the following command:
`(venv) PS C:\Users\TAAHASE0\dev\projects\werkzeugtest>  c:; cd 'c:\Users\TAAHASE0\dev\projects\werkzeugtest'; & 'c:\Users\TAAHASE0\dev\projects\werkzeugtest\venv\Scripts\python.exe' 'c:\Users\TAAHASE0\.vscode\extensions\ms-python.python-2023.6.0\pythonFiles\lib\python\debugpy\adapter/../..\debugpy\launcher' '50758' '--' '-m' 'flask' 'run' '--no-debugger'`

5. Save the app.py file, which causes the werkzeug dev server to reload
` * Detected change in 'C:\\Users\\TAAHASE0\\dev\\projects\\werkzeugtest\\app.py', reloading`

which leads to the before mentioned Exception.

Running the flask app directly from the terminal does not cause this behavior, it reloads the app without error, so it's not an issue of the werkzeug server itself:

(venv) PS C:\Users\TAAHASE0\dev\projects\werkzeugtest> flask run --debug --no-debugger

Diagnostic data

Output for Python in the Output panel (ViewOutput, change the drop-down the upper-right of the Output panel to Python)

``` DAP Server launched with command: c:\Users\TAAHASE0\dev\projects\werkzeugtest\venv\Scripts\python.exe c:\Users\TAAHASE0\.vscode\extensions\ms-python.python-2023.6.0\pythonFiles\lib\python\debugpy\adapter Send text to terminal: c:; cd 'c:\Users\TAAHASE0\dev\projects\werkzeugtest'; & 'c:\Users\TAAHASE0\dev\projects\werkzeugtest\venv\Scripts\python.exe' 'c:\Users\TAAHASE0\.vscode\extensions\ms-python.python-2023.6.0\pythonFiles\lib\python\debugpy\adapter/../..\debugpy\launcher' '53501' '--' '-m' 'flask' 'run' '--no-debugger' Connecting to DAP Server at: 127.0.0.1:53499 Connecting to DAP Server at: 127.0.0.1:53499 ```

User Settings

``` languageServer: "Pylance" linting • flake8Enabled: true • mypyEnabled: true formatting • provider: "black" testing • pytestEnabled: true ```

Extension version: 2023.6.0 VS Code version: Code 1.77.0 (7f329fe6c66b0f86ae1574c2911b681ad5a45d63, 2023-03-29T10:02:16.981Z) OS version: Windows_NT x64 10.0.19042 Modes: Sandboxed: No

System Info |Item|Value| |---|---| |CPUs|Intel(R) Core(TM) i5-8365U CPU @ 1.60GHz (8 x 1896)| |GPU Status|2d_canvas: enabled
canvas_oop_rasterization: disabled_off
direct_rendering_display_compositor: disabled_off_ok
gpu_compositing: enabled
multiple_raster_threads: enabled_on
opengl: enabled_on
rasterization: enabled
raw_draw: disabled_off_ok
skia_renderer: enabled_on
video_decode: enabled
video_encode: enabled
vulkan: disabled_off
webgl: enabled
webgl2: enabled
webgpu: disabled_off| |Load (avg)|undefined| |Memory (System)|15.59GB (3.37GB free)| |Process Argv|--crash-reporter-id 7fc99dfd-d84f-49f3-b924-c4f1ecf5ddd2| |Screen Reader|no| |VM|0%|
A/B Experiments ``` vsliv368cf:30146710 vsreu685:30147344 python383:30185418 vspor879:30202332 vspor708:30202333 vspor363:30204092 vslsvsres303:30308271 vserr242cf:30382550 pythontb:30283811 vsjup518:30340749 pythonptprofiler:30281270 vshan820:30294714 vstes263:30335439 vscoreces:30445986 pythondataviewer:30285071 vscod805cf:30301675 binariesv615:30325510 bridge0708:30335490 bridge0723:30353136 cmake_vspar411:30581797 vsaa593cf:30376535 pythonvs932:30410667 cppdebug:30492333 vsclangdf:30486550 c4g48928:30535728 dsvsc012:30540252 pynewext54:30695312 azure-dev_surveyone:30548225 nodejswelcome1:30587005 282f8724:30602487 pyind779:30671433 f6dab269:30613381 pythonsymbol12:30671437 a9j8j154:30646983 vsccsb:30677849 vscodeenable:30660116 azdwalk:30687957 pythonms35:30701012 ```
paulacamargo25 commented 1 year ago

Hi @sergehauri can you try again with the latest version of the Python extension? I tried to replicate the error, but in my case it works fine. Could you confirm that the error persist? Thanks

sergehauri commented 1 year ago

Hi @paulacamargo25 I just tested on another computer with clean, new Python and VS Code installs and can reproduce the error.

Python version: 3.11.3 VS Code: 1.77.3 Python Extension: 2023.6.0, also tested with Prerelease v2023.7.11011538, same result OS: Windows 11 Pro, Version 10.0.22621 Build 22621 Python packages in venv:

click==8.1.3
colorama==0.4.6
Flask==2.2.3
itsdangerous==2.1.2
Jinja2==3.1.2
MarkupSafe==2.1.2
Werkzeug==2.2.3

There's some additional Debugger warning messages in the output, though I don't think they're of relevance.

(venv) PS G:\My Drive\dev\projects\werkzeugtest>  & 'g:\My Drive\dev\projects\werkzeugtest\venv\Scripts\python.exe' 'c:\Users\serge\.vscode\extensions\ms-python.python-2023.7.11011538\pythonFiles\lib\python\debugpy\adapter/../..\debugpy\launcher' '50153' '--' '-m' 'flask' 'run' '--no-debugger' 
 * Serving Flask app 'app.py'
 * Debug mode: on
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on http://127.0.0.1:5000
Press CTRL+C to quit
 * Restarting with stat
0.03s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
 * Detected change in 'G:\\My Drive\\dev\\projects\\werkzeugtest\\app.py', reloading
Exception in thread Thread-6 (serve_forever):
Traceback (most recent call last):
  File "C:\Users\serge\AppData\Local\Programs\Python\Python311\Lib\threading.py", line 1038, in _bootstrap_inner
    self.run()
  File "C:\Users\serge\AppData\Local\Programs\Python\Python311\Lib\threading.py", line 975, in run
    self._target(*self._args, **self._kwargs)
  File "g:\My Drive\dev\projects\werkzeugtest\venv\Lib\site-packages\werkzeug\serving.py", line 766, in serve_forever
    super().serve_forever(poll_interval=poll_interval)
  File "C:\Users\serge\AppData\Local\Programs\Python\Python311\Lib\socketserver.py", line 233, in serve_forever
    ready = selector.select(poll_interval)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\serge\AppData\Local\Programs\Python\Python311\Lib\selectors.py", line 323, in select
    r, w, _ = self._select(self._readers, self._writers, [], timeout)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\serge\AppData\Local\Programs\Python\Python311\Lib\selectors.py", line 314, in _select
    r, w, x = select.select(r, w, w, timeout)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [WinError 10038] An operation was attempted on something that is not a socket
 * Restarting with stat
0.02s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
nimicohgr commented 1 year ago

I am experiencing the same issue with my Flask application in VS Code. I am running the same versions as @sergehauri in my app, but to uncomplicate things, I created a new project and followed their reproduction instructions and receive the same traceback when debugging following a reload.

Python version: 3.11.3
VS Code: 1.77.3
Python Extension: 2023.6.0, also tested with Prerelease v2023.7.11011538, same result
OS: Windows 11 Pro, Version 10.0.22621 Build 22621
paulacamargo25 commented 1 year ago

Thanks @sergehauri, I will try to reproduce the bug.

schlp commented 1 year ago

facing the same issue, Python3.11.0 and 3.11.4, Flask-2.2.5 and Flask-2.3.2

starball5 commented 1 year ago

Related Werkzeug issue ticket: https://github.com/pallets/werkzeug/issues/2627

Related on Stack Overflow:

anhhaibkhn commented 1 year ago

May I ask if there are any updates on this issue, I have the same problem with Flask 2.3.3, werkzeug 2.3.7, Python 3.11.4

rwmnau commented 10 months ago

Just wanted to add that this is happening to me as well: Flask 3.0.0, Werkzeug 3.0.1, VS Code 1.83.1, and Python extensions 2023.20.0. Same behavior on Python 3.10.6, 3.11.4, and 3.12.0.

Running "flask run --debug" from the command line doesn't have this issue - I can change and save files and it detects the change and restarts with no issue, so it's only while debugging inside VS Code.

olidel001 commented 9 months ago

Hi,

Same here. Exception in thread Thread-1 (serve_forever): Traceback (most recent call last): File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.1776.0_x64__qbz5n2kfra8p0\Lib\threading.py", line 1045, in _bootstrap_inner self.run() File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.1776.0_x64__qbz5n2kfra8p0\Lib\threading.py", line 982, in run self._target(*self._args, **self._kwargs) File "C:\Users\xxxxxxxxxxxxxxxx\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\werkzeug\serving.py", line 806, in serve_forever super().serve_forever(poll_interval=poll_interval) File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.1776.0_x64__qbz5n2kfra8p0\Lib\socketserver.py", line 233, in serve_forever ready = selector.select(poll_interval) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.1776.0_x64__qbz5n2kfra8p0\Lib\selectors.py", line 323, in select r, w, _ = self._select(self._readers, self._writers, [], timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.1776.0_x64__qbz5n2kfra8p0\Lib\selectors.py", line 314, in _select r, w, x = select.select(r, w, w, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ OSError: [WinError 10038]

Thank you

olidel001 commented 9 months ago

My python setup and module list are:

` Flask 3.0.0 Werkzeug 3.0.1 Python 3.11.6

VSCODE: Version: 1.84.1 (system setup) Commit: 2b35e1e6d88f1ce073683991d1eff5284a32690f Date: 2023-11-06T12:35:59.022Z Electron: 25.9.2 ElectronBuildId: 24603566 Chromium: 114.0.5735.289 Node.js: 18.15.0 V8: 11.4.183.29-electron.0 OS: Windows_NT x64 10.0.19045

Python extension pack: v1.7.0 `

rwmnau commented 9 months ago

After a bit of digging, I think the real issue here is that VS Code Python debugger considers a System.Exit with any code other than zero to be an exception and there's no way (currently) to change this functionality (other than ignoring all exceptions, which isn't ideal). When it's running without a debugger, the child process executes the System.Exit(3) and it's caught by the outer process and triggers a restart of the server, all before the other exception (the socket issue) is exposed - but when the debugger is attached (either launched from inside VS Code or launched separately and then attached from VS Code), it stops on the Exit and also allows the other exception to be exposed instead of swallowed. I messed around a bit and can't get code that both swallows this exception and allows the server to restart on file changes as expected.

StackOverflow Post (with people checking in every year since 2018 with "Still not fixed"): https://stackoverflow.com/questions/52372810/visual-studio-code-python-debugging-exception-has-occurred-systemexit

Open DebugPy issue: https://github.com/microsoft/debugpy/issues/1306

nour-s commented 4 months ago

2024, I have the latest of everything from VSCode to Python 3.11 to Flask 3 and even Werkzeug 3.0.2 yet I have the exact issue.

Did all suggestions for Winsock2 fixes and nothing changed.

netsh winsock reset catalog
netsh int ipv4 reset reset.log
netsh int ipv6 reset reset.log
danielniccoli commented 3 months ago

@rwmnau @sergehauri @nimicohgr @olidel001

The behavior of debugpy is not in compliance with the specifications. Because the underlying issue is broader, I raised a separate ticket on the matter, which explains the problem and suggests a solution that should not only fix the issue you are having, but other issues as well. Unforatunately, it was closed and moved to a discussion. Please contribute, if you wish to see it fixed.

The issue: https://github.com/microsoft/debugpy/issues/1590 The discussion: https://github.com/microsoft/debugpy/discussions/1591