Total Disk Space: Total disk space available on the machine (instance had 200 GB).
spaceAvailable: Total disk space configured to be made available for YT-synch service(configured to 100GB)
spaceUsed: Space already used by all the downloaded videos, would be less than spaceAvailable
Right now youtube-synch downloads videos in a batch (of 50, which is configurable) for postprocessing, i.e. merging audio/video, calculating hash, etc. And before start downloading the batch it only ensures that some space is available (i.e. spaceUsed < spaceAvailable)
The problem with the above approach is that the size of the prepared batch might actually exceed not only the availableSpace but all the system disk space too, and it could lead to service crash.
Fix
Pre-calculate the size of the batch & only queue it for downloading if the space would not exceed even after all the videos have been downloaded. i.e. (spaceUsed + batchSize <= spaceAvailable)
Problem
A few definitions first:
spaceAvailableRight now youtube-synch downloads videos in a batch (of 50, which is configurable) for postprocessing, i.e. merging audio/video, calculating hash, etc. And before start downloading the batch it only ensures that some space is available (i.e.
spaceUsed<spaceAvailable)The problem with the above approach is that the size of the prepared batch might actually exceed not only the
availableSpacebut all the system disk space too, and it could lead to service crash.Fix
Pre-calculate the size of the batch & only queue it for downloading if the space would not exceed even after all the videos have been downloaded. i.e. (
spaceUsed+batchSize<=spaceAvailable)