MalloyDelacroix / DownloaderForReddit

The Downloader for Reddit is a GUI application with some advanced features to extract and download submitted content from reddit.
GNU General Public License v3.0
496 stars 47 forks source link

Downloading "pauses" intermittently #348

Closed thany closed 1 year ago

thany commented 1 year ago

Describe the bug Possibly related to #279, but this one is also slightly different. The UX is similar though, in that nothing happens for a while.

While downloading, doesn't matter what, it seems, the application will just sort of sit there. It'll do absolutely nothing for a few minutes and then carry on as if not a cloud in the sky. And when I say nothing, I mean no CPU usage, no disk I/O, no network I/O, and no changes in memory usage.

Upon investigation with Process Hacker, I can see no obvious weirdness happening. Everything is ay-okay. But also nothing going on even if I drill down into the threads or handles. It's sitting absolutely and utterly idle. So then why isn't it carrying on like it's supposed to?

Environment Information

For all but the most trivial of issues, please attach the latest log file. The log doesn't show anything going on. Like I said, nothing happens. Not even a log entry gets added. Except one for being finished when it finally carries on and gets to the end.

MalloyDelacroix commented 1 year ago

This is a rate limit wait time built into PRAW and imposed by reddit. Reddit claims their rate limit is 60 requests per minute, but in my experience, this is not true. Each response from the reddit API that indicates a rate limit infraction will increase the sleep time between requests up to a point. For long download sessions, this can lead to fairly long wait times. Unfortunately, there is nothing I can do about this. Reddit does not even disclose what their real rate limit policy is.

We increase the rate limit by allowing users to authenticate through reddit. Users that are authenticated will see less of a wait time between requests.

thany commented 1 year ago

Alright thanks 🙂

Maybe add an entry in the log when rate limiting is ocurring, especially when verbose logging is enabled. I think it's a good idea to let users know what the program is currently doing (or not doing in this case). It might prevent issues/questions like this in the future.

MalloyDelacroix commented 1 year ago

That's a good idea, I should put that in the log.