ytdl-org / youtube-dl

Command-line program to download videos from YouTube.com and other video sites
http://ytdl-org.github.io/youtube-dl/
The Unlicense
132.39k stars 10.04k forks source link

Multiple youtube-dl instances downloading the same video causes malformed output #21449

Closed sharinq closed 5 years ago

sharinq commented 5 years ago

Checklist

Verbose log

Description

Note: I've only tested this with Windows and NHK Video on Demand videos.

When multiple instances of youtube-dl are downloading the same file at the same time, it can cause output files with repeated parts.

For example, this command downloads a 25,406 KB (26,014,882 bytes) video which is 3:00 long.

youtube-dl https://www3.nhk.or.jp/nhkworld/en/ondemand/video/6022045/

However, if I run it from two command line windows at the same time, it will lead to the error [WinError 32] The process cannot access the file because it is being used by another process, but if I continue resuming (and interrupting) the processes to give them more time to try to download the same video at the same time, the output videos are larger with short pieces of the video repeated. I believe I've experienced this in the past causing the entire video to be downloaded multiple times and joined together, but I haven't replicated it.

The reason for wanting to not have multiple instances of youtube-dl cause malformed output is that I'm using a script to collect video URLs to download and then giving them to youtube-dl, and this is scheduled to run using Windows Task Scheduler. If the last task is still running when it is time for the next task to be run, the script will be run again and youtube-dl might try to download the same video at the same time.

Is there any option that could help with this?

dstftw commented 5 years ago

This is expected behavior. You must not download to the same file simultaneously in the first place.