Open pawelmalak opened 6 months ago
development
--max-wait-time option is being ignored. I've set it to 10, but it still waits 60 seconds on each error.
--max-wait-time
bdfr download \ "data/subreddits/$1" \ --config config.cfg \ --subreddit $1 \ --sort top \ --time year \ --verbose \ --skip html \ --skip txt \ --skip-domain youtube.com \ --skip-domain youtu.be \ --skip-domain vimeo.com \ --skip-domain archive.org \ --file-scheme "{DATE}_{TITLE}_{POSTID}" \ --folder-scheme . \ --exclude-id-file "data/ids/ids-$1.txt" \ --log "data/logs/year_$1.log" \ --max-wait-time 10
In the command --max-wait-time is set to 10.
Local config.cfg file which is passed to the command also has max_wait_time set to 10:
config.cfg
max_wait_time
[DEFAULT] client_id = <redacted> client_secret = <redacted> scopes = identity, history, read, save, mysubreddits backup_log_count = 3 max_wait_time = 10 time_format = ISO user_token = <redacted>
Deafult default_config.cfg at C:\Users\Admin\AppData\Local\BDFR\bdfr also has it set to 10:
default_config.cfg
C:\Users\Admin\AppData\Local\BDFR\bdfr
[2024-03-09 13:41:27,586 - bdfr.downloader - DEBUG] - Attempting to download submission X1 [2024-03-09 13:41:27,588 - bdfr.downloader - DEBUG] - Using Imgur with url http://imgur.com/U1 [2024-03-09 13:41:28,083 - bdfr.resource - WARNING] - Error occured downloading from https://i.imgur.com/U1, waiting 60 seconds: Response code 429 [2024-03-09 13:42:28,088 - bdfr.resource - ERROR] - Max wait time exceeded for resource at url https://i.imgur.com/U1 [2024-03-09 13:42:28,088 - bdfr.downloader - ERROR] - Failed to download resource https://i.imgur.com/U1 in submission X1 with downloader Imgur: Could not download resource: Response code 429 [2024-03-09 13:42:28,089 - bdfr.downloader - DEBUG] - Attempting to download submission X2 [2024-03-09 13:42:28,091 - bdfr.downloader - DEBUG] - Using Imgur with url http://imgur.com/U2 [2024-03-09 13:42:28,813 - bdfr.resource - WARNING] - Error occured downloading from http://imgur.com/U2, waiting 60 seconds: Response code 429 [2024-03-09 13:43:28,825 - bdfr.resource - ERROR] - Max wait time exceeded for resource at url http://imgur.com/U2 [2024-03-09 13:43:28,825 - bdfr.downloader - ERROR] - Failed to download resource http://imgur.com/U2 in submission X2 with downloader Imgur: Could not download resource: Response code 429 [2024-03-09 13:43:28,826 - bdfr.downloader - DEBUG] - Attempting to download submission X3 [2024-03-09 13:43:28,827 - bdfr.downloader - DEBUG] - Using Imgur with url http://imgur.com/U3 [2024-03-09 13:43:29,560 - bdfr.resource - WARNING] - Error occured downloading from http://imgur.com/U3, waiting 60 seconds: Response code 429 [2024-03-09 13:44:29,570 - bdfr.resource - ERROR] - Max wait time exceeded for resource at url http://imgur.com/U3 [2024-03-09 13:44:29,570 - bdfr.downloader - ERROR] - Failed to download resource http://imgur.com/U3 in submission X3 with downloader Imgur: Could not download resource: Response code 429
As you can see, on every error, BDfR waits for 60 seconds: Error occured downloading from URL, waiting 60 seconds
Error occured downloading from URL, waiting 60 seconds
Same issue, is this fork dead?
It's not dead, I've just been busy with the semester of late. I will try to get to this soon.
Same issue :/
development
branch)Description
--max-wait-time
option is being ignored. I've set it to 10, but it still waits 60 seconds on each error.Command
In the command
--max-wait-time
is set to 10.Local
config.cfg
file which is passed to the command also hasmax_wait_time
set to 10:Deafult
default_config.cfg
atC:\Users\Admin\AppData\Local\BDFR\bdfr
also has it set to 10:Environment (please complete the following information)
development
branchLogs
As you can see, on every error, BDfR waits for 60 seconds:
Error occured downloading from URL, waiting 60 seconds