CDL checks free space on the drive before starting a new file download. However, when running with -retry-failed, it will download to the original download folder used when the URL was added to the DB, not the currently specified OUTPUT_FOLDER. If that folder and OUTPUT_FOLDER are on different drives, CDL will keep downloading until no space is left (0B)
Use CDL to download a file bigger than the RAMDISK (URLS.txt with example attached) and use the config folder created.
The download will be skipped (CDL requires 5GB free by default) but it will create the default global_settings.yaml
Edit global_settings.yaml and change required_free_space to 1GB
sed -i 's@required_free_space: 5@required_free_space: 1@g' /tmp/CDL_test_config/AppData/Configs/global_settings.yaml
Run CDL a second time with the same parameters but cancel the download (CTRL + C) at about 600MB downloaded, when the RAMDISK has less than 1GB available. This will also add the file as incomplete (failed) to the database
Run the command a third time. In this case, the download will be skipped by config cause RAMDISK does not have 1GB available anymore. That's the normal behavior
Run CDL one last time but now change the OUTPUT_FOLDER to any folder on a different drive, and add the flag --retry-failed.
Download will start as normal cause CDL checked free space on /different/download/path, not on the RAMDISK. However, the file is still downloaded to the RAMDISK.
The download will continue until the drive is completely out of space. CDL will not crash but the download will just fail with "Unknown" as error on the UI, but the logs do show "OSError: [Errno 28] No space left on device". If the database is on that drive, sometimes it gets corrupted. Probably because CDL tries to write to it at the end.
CDL checks free space on the drive before starting a new file download. However, when running with
-retry-failed
, it will download to the original download folder used when the URL was added to the DB, not the currently specifiedOUTPUT_FOLDER
. If that folder andOUTPUT_FOLDER
are on different drives, CDL will keep downloading until no space is left (0B)Setup Information
How to reproduce (UNIX only):
global_settings.yaml
global_settings.yaml
and changerequired_free_space
to 1GBRun CDL a second time with the same parameters but cancel the download (
CTRL
+C
) at about 600MB downloaded, when the RAMDISK has less than 1GB available. This will also add the file as incomplete (failed) to the databaseRun the command a third time. In this case, the download will be
skipped by config
cause RAMDISK does not have 1GB available anymore. That's the normal behaviorRun CDL one last time but now change the
OUTPUT_FOLDER
to any folder on a different drive, and add the flag--retry-failed
./different/download/path
, not on the RAMDISK. However, the file is still downloaded to the RAMDISK.The download will continue until the drive is completely out of space. CDL will not crash but the download will just fail with "Unknown" as error on the UI, but the logs do show "
OSError: [Errno 28] No space left on device"
. If the database is on that drive, sometimes it gets corrupted. Probably because CDL tries to write to it at the end.Attachments:
downloader.log URLs.txt