search

Serene-Arc / bulk-downloader-for-reddit

Downloads and archives content from reddit
https://pypi.org/project/bdfr
GNU General Public License v3.0
2.31k stars 213 forks source link

[BUG] PRAW Error Code 429 #910

Open GarethFreeman opened 1 year ago

GarethFreeman commented 1 year ago

Description

I am trying to rip a reddit user's submitted posts but keep getting blocked by PRAW, I've also tried updating with no success. It does work sometimes but it much slower than usual.

Command

py -3 -m bdfr download C:\bdfrips\tti --user TameTheseImpalas --submitted

Environment (please complete the following information)

Logs

py -3 -m bdfr download C:\bdfrips\tti --user TameTheseImpalas --submitted [2023-07-11 19:55:00,020 - bdfr.connector - ERROR] - User TameTheseImpalas failed to be retrieved due to a PRAW exception: received 429 HTTP response

twentyonerooms87 commented 1 year ago

Thanks Serene-Arc! That did the trick!

Serene-Arc commented 1 year ago

Hopefully the development has the 429 error fixed, but I won't be able to merge it into master until Ali changes the secrets. I don't have the power to override the master branch protections, so the tests HAVE to be passing before it can be merged. At the moment, The repository secrets have the old client ID which means that the tests take too long and time out. I don't have the power to change that either.

For everyone coming here, pull and use the development branch for the moment and I'll get it merged to master as soon as possible.

MachiavelliSeraphim commented 1 year ago

Not to sound like an idiot, but ive never pulled from a development branch with pip before. Is there a command/switch to have pip pull the development branch instead of releases?

Serene-Arc commented 1 year ago

You have to specify the link and then pip will handle it. The command that you want is this:

pip install -U git+https://github.com/aliparlakci/bulk-downloader-for-reddit.git@development

For anyone wondering about updates, I've contacted Ali and am waiting for him to respond. Unfortunately this is as much as I can do at the moment.

MachiavelliSeraphim commented 1 year ago

Updated to the development branch and now I get "[2023-09-18 06:50:22,315 - bdfr.connector - WARNING] - Using an unauthenticated app like this will result in Reddit limiting queries to 10 requests a minute" for some odd reason.... Config file is manually specified with correct client and secret. And BDFR seems to be running fine at full speed which is more than 10/minute....

slitchampion commented 11 months ago

Hopefully the development has the 429 error fixed, but I won't be able to merge it into master until Ali changes the secrets. I don't have the power to override the master branch protections, so the tests HAVE to be passing before it can be merged. At the moment, The repository secrets have the old client ID which means that the tests take too long and time out. I don't have the power to change that either.

For everyone coming here, pull and use the development branch for the moment and I'll get it merged to master as soon as possible.

Any luck getting ahold of Ali?

Serene-Arc commented 10 months ago

Not as of yet unfortunately