MalloyDelacroix / DownloaderForReddit

The Downloader for Reddit is a GUI application with some advanced features to extract and download submitted content from reddit.
GNU General Public License v3.0
496 stars 47 forks source link

Getting more crashes since Reddit API change. #354

Open LevelTwelve opened 11 months ago

LevelTwelve commented 11 months ago

Describe the bug Since the Reddit API change getting more application crashes on download Environment Information

When I download a user list the app will crash and produce this result in the log.

"levelname": "CRITICAL",
"asctime": "07/14/2023 11:25:19 AM",
"filename": "main.py",
"module": "main",
"name": "DownloaderForReddit.__main__",
"funcName": "log_unhandled_exception",
"lineno": 48,
"message": "Unhandled exception",
"exc_info": "Traceback (most recent call last):\n  File \"DownloaderForReddit\\core\\download_runner.py\", line 182, in run\n  File \"DownloaderForReddit\\core\\download_runner.py\", line 237, in run_download\n  File \"DownloaderForReddit\\core\\runner.py\", line 12, in check\n  File \"DownloaderForReddit\\core\\download_runner.py\", line 277, in get_user_submissions\n  File \"DownloaderForReddit\\core\\runner.py\", line 12, in check\n  File \"DownloaderForReddit\\core\\download_runner.py\", line 283, in get_user_submissions\n  File \"DownloaderForReddit\\core\\download_runner.py\", line 298, in handle_submissions\n  File \"DownloaderForReddit\\core\\runner.py\", line 12, in check\n  File \"DownloaderForReddit\\core\\download_runner.py\", line 322, in get_submissions\n  File \"praw\\models\\listing\\generator.py\", line 63, in __next__\n  File \"praw\\models\\listing\\generator.py\", line 73, in _next_batch\n  File \"praw\\reddit.py\", line 595, in get\n  File \"praw\\reddit.py\", line 696, in _objectify_request\n  File \"praw\\reddit.py\", line 885, in request\n  File \"prawcore\\sessions.py\", line 330, in request\n  File \"prawcore\\sessions.py\", line 266, in _request_with_retries\nprawcore.exceptions.TooManyRequests: received 429 HTTP response"

}

MelanieTanaka commented 11 months ago

I had the exact same issue, work around is to log in a reddit account.

I think it has to do with the new API nonsense. The app should probably be updated to catch this error and tell the user what to do. I don't think there's any other fix on the developer side.

LevelTwelve commented 11 months ago

I did connect Reddit account under File - Connect Reddit Account. Proceeded to download and received the same error in the log.

{ "levelname": "CRITICAL", "asctime": "07/16/2023 12:05:10 PM", "filename": "main.py", "module": "main", "name": "DownloaderForReddit.main", "funcName": "log_unhandled_exception", "lineno": 48, "message": "Unhandled exception", "exc_info": "Traceback (most recent call last):\n File \"DownloaderForReddit\core\download_runner.py\", line 182, in run\n File \"DownloaderForReddit\core\download_runner.py\", line 237, in run_download\n File \"DownloaderForReddit\core\runner.py\", line 12, in check\n File \"DownloaderForReddit\core\download_runner.py\", line 277, in get_user_submissions\n File \"DownloaderForReddit\core\runner.py\", line 12, in check\n File \"DownloaderForReddit\core\download_runner.py\", line 283, in get_user_submissions\n File \"DownloaderForReddit\core\download_runner.py\", line 298, in handle_submissions\n File \"DownloaderForReddit\core\runner.py\", line 12, in check\n File \"DownloaderForReddit\core\download_runner.py\", line 322, in get_submissions\n File \"praw\models\listing\generator.py\", line 63, in next\n File \"praw\models\listing\generator.py\", line 73, in _next_batch\n File \"praw\reddit.py\", line 595, in get\n File \"praw\reddit.py\", line 696, in _objectify_request\n File \"praw\reddit.py\", line 885, in request\n File \"prawcore\sessions.py\", line 330, in request\n File \"prawcore\sessions.py\", line 266, in _request_with_retries\nprawcore.exceptions.TooManyRequests: received 429 HTTP response"

QbiX90 commented 11 months ago

image Something is wrong with the connnect Reddit Account feature. Its logged out after exe restart and never logs in correctly.

MelanieTanaka commented 11 months ago

I unfortunately can't help any more than that, I'm not involved in the development. I had the same error in my logs and crashing when download process would start, and solved it as described. I found this error report when googling my logs, then remembered the API changes of reddit and decided to try adding an account.

Maybe something more complex in your case, or are you on the latest release?

MalloyDelacroix commented 11 months ago

I can't reproduce these errors consistently enough to diagnose the problem. Its obviously a result of reddit's API changes, but I'm not sure exactly how to deal with it. They seem to be very sporadic.

Since this issue has been posted, I have registered the app through reddit's new registration system. That does seem to have improved the performance.

I think the issue may be that since applications are now only allowed a certain number of API calls under the free tier, this app may be hitting the limit on some days. This would explain the other issue in this thread about not being able to connect to a reddit account. Once so many API calls are made, reddit starts returning 429 responses for all calls.

MelanieTanaka commented 11 months ago

I can't reproduce these errors consistently enough to diagnose the problem. Its obviously a result of reddit's API changes, but I'm not sure exactly how to deal with it. They seem to be very sporadic.

Since this issue has been posted, I have registered the app through reddit's new registration system. That does seem to have improved the performance.

I think the issue may be that since applications are now only allowed a certain number of API calls under the free tier, this app may be hitting the limit on some days. This would explain the other issue in this thread about not being able to connect to a reddit account. Once so many API calls are made, reddit starts returning 429 responses for all calls.

Is there maybe some way to let each user optionally use their own API key, and default to the one you registered for those who don't bother? I'm guessing that'll probably have to become the norm at some point with everything reddit is doing.

MalloyDelacroix commented 11 months ago

I am considering that solution. Though my thoughts are it would be a short term solution. Reddit has to see that coming and have measures in place or underway to prevent it.