pukkandan
commented
3 years ago As far as I know, os.remove is a synchronous operation and should return only after the file is actually deleted. I suspect there is something else going on that is causing the issue
Checklist
Verbose log
LOG A:
LOG B:
Username was edited, replaced with 'user_name'. Descriptions and Comments in metadata shortened (unnecessary text). LOG A an example of output I got. LOG B has debug info.
Description
As far as I know, this happens only when downloading a PL playlist. Have not tried other forms of downloading, such as multiple single videos. Still it is possible to happen in any download that uses ffmpeg.py Problem: When downloading a complete playlist from youtube, I would get the error seen in LOG A and LOG B. It happens randomly. It would download 2-5 videos, then on the 5th video the error occurred. FileExistsError: [WinError 183] Repeated tries gets the same error at the same video. I would delete the 5th video (the one just downloaded when error triggered), and retry. With luck, download continues normally until on the 16th video the same error occurs. Rinse and repeat.
What seems to happen: From what I saw, this is what youtube-dlc does:
Where the problem occurrs: In the step 4 that I just listed is where the error occurrs. Looking at traceback of the error, at ffmpeg.py, line 536 (os.rename) is the culprit.
What I believe is happening (on step 4 I just listed above): On line 535 of ffmpeg.py is os.remove. It begins to remove the video1.mp4 file. However, while the remove command was received by Windows, and execution has begun, next line is being interpreted by Python. Next line is 536, os.rename, which will rename video1.temp.mp4 to video1.mp4. However, when rename command is given to Windows, the remove command is still executing, meaning video1.mp4 still exists. As such, rename command begin executing, but sees that video1.mp4 already exists, and cancel the renaming raising the FileExistsError.
Thus for os.rename, os.remove is too slow to complete, and for os.remove, os.rename is too fast to execute.
A quick search of os.remove in ffmpeg.py reveals that the same thing could happen in several classes where os.remove is in line n and os.rename is in line n+1.
My fix: Because it seems to be a delay in the execution of a command when another command is asked to execute, I just added a time.sleep between the os.remove and os.rename.
Adding that line solved my problem. Used pyinstaller to get executable. Downloaded the complete playlist without a problem.
Unique problem? Unique circumstances? While I think that if it happened to me, it could happen to anyone else too, it could be because of the environment. My computer is 10 years old. Old hardware tend to get slow over time. Running processes could also affect, as I had browser open with multiple tabs and was watching a stream. When deleting an error-causing downloaded video, at the confirmation dialog I saw another dialog behind that seemed to be processing a regular delete-a-file command (I use SHIFT+DEL for deleting) that never completed. Which is why I think os.remove is too slow and os.rename is too quick.
My sugestions to solve the bug:
I did searched on github for similar problems. I found one addressing os.rename in ffmpeg.py, but it was not this problem I found. My understanding of Python is basic (have coded only few simple scripts for learning the basic concepts.) Working on a solution myself is currently not possible. I hope this help in creating a solution.