pyinstaller temp files are not cleaned up when using yt-dlp.exe with mpv

skotarek commented 5 months ago

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

[X] I'm asking a question and not reporting a bug or requesting a feature
[X] I've looked through the README
[X] I've verified that I have updated yt-dlp to nightly or master (update instructions)
[X] I've searched known issues and the bugtracker for similar questions including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue

Please make sure the question is worded well enough to be understood

temp

as far as I understand the files in the %temp% folder generated by yt-dlp.exe are supposed to automatically delete themselves, however only some of them do and most remain on my pc taking up gigabytes of space

I think this happens most often when I'm running an mpv playlist as I haven't noticed them not being deleted when running a single streamed mpv video but I'm not entirely sure about that

what can I possibly do here? what could be the cause and what is the solution? I'm looking for a more in-depth explanation rather than just a workaround like creating a batch file to remove these files and running it on startup

UPDATE: after further observations I came to believe that this problem only occurs when I skip to the next video before current one ends its startup loading - perhaps the cleaning function gets stopped because of it?

Provide verbose output that clearly demonstrates the problem

[ ] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[ ] If using API, add 'verbose': True to YoutubeDL params instead
[ ] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

No response

bashonly commented 5 months ago

This issue was also reported to mpv ~2 years ago:

https://github.com/mpv-player/mpv/issues/10747

yt-dlp seems to create a pyinstaller folder _MEI<random_numbers> in %TMP% every time a network stream is played (not youtube) in mpv. Apprently it's the expected behaviour with exe version as mentioned in this isssue Is it possible for the .zip version to work with mpv or will only the exe version work as yt-dlp seems to create the exactly same files every time and not auto deleting it.

But since the reporter asked the wrong question in the title of the issue, that is the only question that was addressed, and the issue was closed.

The temp files are coming from pyinstaller, which is what yt-dlp uses to bundle (most of) its Windows executables. It needs to unpack the libraries and script files before it can execute yt-dlp. It should be cleaning up after itself, but maybe something is going wrong when mpv calls it.

yt-dlp devs won't be able to fix this; it would be something for the devs of mpv or maybe pyinstaller to fix.

Your alternatives are to install with pip or use the Windows onedir bundle. The caveat with these alternatives is that you won't be able to use yt-dlp's self-updater (-U/--update-to). If you've installed with pip, then you'll need to upgrade using pip, and if using the onedir bundles then you'll need to manually download the new releases.

Another alternative is using the py2exe build, but this is not recommended since it is missing some functionality and will eventually/soon be discontinued.

geextahslex commented 5 months ago

This is a good question, I was wondering why I lost 6,5GB of my C: drive. And that was the issue. What is weird that windows clean up doesn't show this in the "remove temporary files" section. He showed only 400MB

As a workaround you could create a batch script that would delete all files in the temp folder that have "_MEI" in their name. And run it at startup

skotarek commented 5 months ago

you could create a batch script that would delete all files in the temp folder that have "_MEI" in their name

I know but this doesn't really fix the issue, it's just a workaround

bashonly commented 5 months ago

As I hinted above, if you want this to be fixed, you'll need to report it on either the mpv bug tracker or pyinstaller bug tracker.

If you can reproduce the issue without mpv, then it should be reported to pyinstaller. If you can only repro with mpv, then it should be reported to mpv (and make sure to reference https://github.com/mpv-player/mpv/issues/10747)

geextahslex commented 5 months ago

This is the batch script that you could put into your autostart folder or make a scheduled task, it will delete all files inside your temp folder that have a "_MEI..." in their name

@ECHO OFF
 FOR /F "TOKENS=*" %%V IN ('DIR /AD /B /S "%TEMP%\_MEI*"') DO RD /S /Q "%%~V"

bashonly commented 5 months ago

Related(?):

bashonly commented 4 months ago

It looks like this may have been fixed today by https://github.com/pyinstaller/pyinstaller/pull/8648

Once pyinstaller releases a new version, I'll update the build workflow to use it and we can see.

bashonly commented 2 months ago

I think we've done all we can on our end with #10709 merged, short of abandoning PyInstaller "onefile" builds altogether

Anyone affected by this issue can give the latest nightly version a try

skotarek commented 2 months ago

I just tried the new nightly version and nothing has changed, the problem still occurs

Jules-A commented 1 month ago

Not only does 6.10.0 fail to delete temp files, it seems to cause yt-dlp sub-processes to stop working entirely in new Python builds with

[PYI-14028:WARNING] Failed to remove temporary directory: C:\Users\Jules\AppData\Local\Temp\_MEI140282
Press any key to continue . . . Traceback (most recent call last):
  File "yt_dlp\__main__.py", line 14, in <module>
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "PyInstaller\loader\pyimod02_importers.py", line 378, in exec_module
  File "yt_dlp\__init__.py", line 18, in <module>
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "PyInstaller\loader\pyimod02_importers.py", line 378, in exec_module
  File "yt_dlp\cookies.py", line 23, in <module>
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "PyInstaller\loader\pyimod02_importers.py", line 378, in exec_module
  File "yt_dlp\aes.py", line 5, in <module>
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "PyInstaller\loader\pyimod02_importers.py", line 378, in exec_module
  File "yt_dlp\dependencies\__init__.py", line 82, in <module>
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "PyInstaller\loader\pyimod02_importers.py", line 378, in exec_module
  File "yt_dlp\dependencies\Cryptodome.py", line 19, in <module>
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "PyInstaller\loader\pyimod02_importers.py", line 378, in exec_module
  File "Cryptodome\Cipher\__init__.py", line 27, in <module>
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "PyInstaller\loader\pyimod02_importers.py", line 378, in exec_module
  File "Cryptodome\Cipher\_mode_ecb.py", line 35, in <module>
  File "Cryptodome\Util\_raw_api.py", line 315, in load_pycryptodome_raw_lib
OSError: Cannot load native module 'Cryptodome.Cipher._raw_ecb': Not found '_raw_ecb.cp312-win_amd64.pyd', Not found '_raw_ecb.pyd'
[PYI-13144:ERROR] Failed to execute script '__main__' due to unhandled exception!

and in the official build just gives a warning

[PYI-12936:WARNING] Failed to remove temporary directory: C:\Users\Jules\AppData\Local\Temp\_MEI129362

but doesn't crash the process it seems.

EDIT: I probably should clarify, this is not when using MPV.

gruiyuan commented 1 month ago

same with https://github.com/yt-dlp/yt-dlp/issues/10294#issuecomment-2401185445 on 6.10.0

but on 6.9.0 it works normally

yt-dlp / yt-dlp