_MEI Folder not deleted for console=True while using windows terminal

stranger0612 commented 2 weeks ago

Description of the issue

In the past i compiled some simple scripts with console=True. Since I'm using windows 11, those executables will be launched in the window terminal by default. After closing the application with the close button of the terminal, I noticed that the _MEI folder won't be deleted anymore. But when I setup my system to use the console host (conhost.exe) and close the same application with the close button, the _MEI folder will be deleted as expected.

I don't know if its a bug or more a user specific problem because the application works, but the cleanup of _MEI folder seems to be broken.

Context information (for bug reports)

Output of pyinstaller --version: 6.8.0
Version of Python: 3.9.13
Platform: Win11
How you installed Python: python.org/downloads
Did you also try this on another platform? Does it work there? I ran the same application while using windows 10 which uses the console host. As expected, it worked and the _MEI folder got deleted
try the latest development version, using the following command:

pip install https://github.com/pyinstaller/pyinstaller/archive/develop.zip

follow all the instructions in our "If Things Go Wrong" Guide (https://github.com/pyinstaller/pyinstaller/wiki/If-Things-Go-Wrong) and

Make sure everything is packaged correctly

[x] start with clean installation
[x] use the latest development version
[x] Run your frozen program from a command window (shell) — instead of double-clicking on it
[ ] Package your program in --onedir mode
[x] Package without UPX, say: use the option --noupx or set upx=False in your .spec-file
[ ] Repackage you application in verbose/debug mode. For this, pass the option --debug to pyi-makespec or pyinstaller or use EXE(..., debug=1, ...) in your .spec file.

A minimal example program which shows the error

python file

print("hello world")

input("press enter to close app...")

spec file

a = Analysis(
    ['hello_world.py'],
    pathex=[],
    binaries=[],
    datas=[],
    hiddenimports=[],
    hookspath=[],
    hooksconfig={},
    runtime_hooks=[],
    excludes=[],
    noarchive=False,
    optimize=0,
)
pyz = PYZ(a.pure)

exe = EXE(
    pyz,
    a.scripts,
    a.binaries,
    a.datas,
    [],
    name='hello_world',
    debug=False,
    bootloader_ignore_signals=False,
    strip=False,
    upx=False,
    upx_exclude=[],
    runtime_tmpdir=None,
    console=True,
    disable_windowed_traceback=False,
    argv_emulation=False,
    target_arch=None,
    codesign_identity=None,
    entitlements_file=None,
)

Stacktrace / full error message

no error message available

Please also see https://github.com/pyinstaller/pyinstaller/wiki/How-to-Report-Bugs for more about what would use to solve the issue.

rokm commented 1 week ago

Hmm, indeed.

I suppose Windows Terminal (which is probably implied by Win11) was also the missing link in OP of https://github.com/pyinstaller/pyinstaller/issues/8571.

nitial analysis suggests that this is due to different behavior of Windows Terminal w.r.t. conhost.

With conhost, both the parent launcher process and its child (main application) process receive CTRL_CLOSE_EVENT immediately when console is to be closed. The event terminates the child process (unless it has its own handler installed), so all we had to do in the parent's handler was to delay the return from handler "forever" (the OS runs this handler is separate thread, so the main thread detected that child has finished/died, performed the cleanup, and exited).

With Windows Terminal, only the parent launcher process seems to receive CTRL_CLOSE_EVENT ; so after the OS-imposed timeout, it is forcibly terminated. And sometime later, it seems that event is also sent to the child process.

I suppose we'll need to rework the logic here, and if the parent receives the event, it should forcibly terminate the child after a short grace period, so it can perform the cleanup.

stranger0612 commented 1 week ago

First of all, thanks for your detailed information about what went wrong.

So as a workaround, I have to make sure, to use conhost whenever I launch a tool that uses console=True.

rokm commented 1 week ago

First of all, thanks for your detailed information about what went wrong.

So as a workaround, I have to make sure, to use conhost whenever I launch a tool that uses console=True.

For now, yes.

stranger0612 commented 2 days ago

First of all, thanks for this quick fix :)

Unfortunately the cleanup still will not work while using window terminal. I just checked the new version 6.9.0

I also used

pip install https://github.com/pyinstaller/pyinstaller/archive/develop.zip

to get the latest development version.

rokm commented 1 day ago

You need to rebuild the bootloader, too...

pip uninstall pyinstaller
set PYINSTALLER_COMPILE_BOOTLOADER=1
pip install https://github.com/pyinstaller/pyinstaller/archive/develop.zip

(assuming you are using command prompt - the syntax for setting environment variable in PowerShell is different).

stranger0612 commented 1 day ago

Thanks for the hint, sorry, didn't know that I have to rebuild the bootloader as well...

With your additional information about the rebuild it works as expected. Once again, thanks for the quick fix :)

So, I have to perform this with every install of pyinstaller? Or will it be available in near future so I can simply use

pip install pyinstaller

rokm commented 1 day ago

So, I have to perform this with every install of pyinstaller?

Only if you try to install from develop and the bootloader code has changed since last release (so if you are installing from develop, it is safest to always force bootloader rebuild as well).

We rebuild and commit bootloaders before each release, so if you do pip install pyinstaller, you get pre-built bootloaders (for the version that you are installing).

Right after release, you can safely install from develop without rebuilding the bootloader, too (until the bootloader code changes).

stranger0612 commented 1 day ago

In summary, with the next release, it should be fine using pip install.

But for now, I have to make sure using the latest development build and rebuild the bootloader to make sure, the cleanup works

pyinstaller / pyinstaller