Redgifs posts fail to download - "Could not read the page source" [BUG]

[germyparker]

• [ x] I am reporting a bug.
• [ x] I am running the latest version of BDfR
• [ x] I have read the Opening an issue

Description

Every redgifs post fails to download:

[2021-06-28 10:01:57,729 - bdfr.downloader - ERROR] - Site Redgifs failed to download submission o9bzms: Could not read the page source

NFSW post

Command

python3.9 -m bdfr download --user [USERNAME] --upvoted --authenticate -L $1 --file-scheme '{SUBREDDIT}_{REDDITOR}_{TITLE}_{POSTID}' ./output/

Environment (please complete the following information):

• OS: MacOS
• Python version: 3.9.4

Logs

No logs other than the output posted above

[Serene-Arc]

I highly doubt that there are no other logs than that single line. Please post all logs from the section is question. It's there for a reason so I can get a full picture of what is happening.

[germyparker]

There really isn't - a while ago BDFR used to create a log file in the output folder with detailed logs in it, but it no longer does that, so all I have are the cli lines, which has: the failed line(s), any successful lines ( eg [2021-06-28 22:25:53,546 - bdfr.downloader - INFO] - Downloaded submission o9jqle from tightdresses ) and then the end line, [2021-06-28 22:25:54,352 - root - INFO] - Program complete

Is there a new location for verbose log files?

[Serene-Arc]

Yes. When you created this bug issue, you checked a box that said you read the instructions on opening a bug. There are clear instructions on where to find the log file and what to provide. In future, don't just tick that box when you obviously haven't done it. It's there for a reason.

Please provide those log files as requested in the template.

[germyparker]

Sorry, I'm not trying to antagonize you - I've been using BDFR for a few years now, and since the v2 changeover, I haven't seen logs in the output, so I figured they were broken. I didn't realize they had moved.

[2021-06-28 22:50:17,728 - bdfr.downloader - DEBUG] - Setting maximum download wait time to 120 seconds
[2021-06-28 22:50:17,729 - bdfr.downloader - Level 9] - Created download filter
[2021-06-28 22:50:17,729 - bdfr.downloader - Level 9] - Created time filter
[2021-06-28 22:50:17,729 - bdfr.downloader - Level 9] - Created sort filter
[2021-06-28 22:50:17,729 - bdfr.downloader - Level 9] - Create file name formatter
[2021-06-28 22:50:17,729 - bdfr.downloader - DEBUG] - Using authenticated Reddit instance
[2021-06-28 22:50:17,730 - bdfr.downloader - Level 9] - Created site authenticator
[2021-06-28 22:50:17,730 - bdfr.downloader - Level 9] - Retrieved subreddits
[2021-06-28 22:50:17,730 - bdfr.downloader - Level 9] - Retrieved multireddits
[2021-06-28 22:50:17,730 - bdfr.oauth2 - Level 9] - Loaded OAuth2 token for authoriser
[2021-06-28 22:50:17,962 - bdfr.oauth2 - Level 9] - Written OAuth2 token from authoriser to /Users/USERNAME/Library/Application Support/bdfr/default_config.cfg
[2021-06-28 22:50:18,198 - bdfr.downloader - DEBUG] - Retrieving upvoted posts of user USERNAME
[2021-06-28 22:50:18,198 - bdfr.downloader - Level 9] - Retrieved user data
[2021-06-28 22:50:18,198 - bdfr.downloader - Level 9] - Retrieved submissions for given links
[2021-06-28 22:50:18,379 - bdfr.downloader - DEBUG] - Attempting to download submission o9yyfd
[2021-06-28 22:50:18,381 - bdfr.downloader - DEBUG] - Using Redgifs with url https://redgifs.com/watch/leafysaltydungbeetle
[2021-06-28 22:50:18,575 - bdfr.downloader - ERROR] - Site Redgifs failed to download submission o9yyfd: Could not read the page source
[2021-06-28 22:50:18,575 - root - INFO] - Program complete

[Serene-Arc]

Thank you. I'll investigate and see what the problem is. At a glance, it may be related to some of the issues listed in #472; a number of Redgifs submissions are unavailable using the API that the BDFR uses.

[Serene-Arc]

This link seems to work for me, with the most recent BDFR code. Is this still an issue for you?

[germyparker]

Yes! I just tried it with about 20 different redgif links and all were successful. Thank you!