search

MalloyDelacroix / DownloaderForReddit

The Downloader for Reddit is a GUI application with some advanced features to extract and download submitted content from reddit.
GNU General Public License v3.0
496 stars 47 forks source link

Banned by RedGifs (Need Proxy Support) #378

Open Jmanfrom87 opened 4 months ago

Jmanfrom87 commented 4 months ago

Keep getting banned by Redgifs.

It looks to be a time ban as it didnt work for few hours and then started working again. Looks like there is a rate limit situation when scraping.

magdev2022 commented 4 months ago

I can help you. @Jmanfrom87

Jmanfrom87 commented 4 months ago

@magdev2022

I am ready for help!

MalloyDelacroix commented 4 months ago

I thought it was only me facing me this problem. Did reddit have daily limit or monthly? If daily perhaps the application can schedule download limit to prevent crash?

The main problem for me is Redgifs. I seem to get banned before scraping even one sub. I tried a VPN and configured the program to only use 1 thread and I still got banned before finishing a sub.

The program is great but we really need rotating proxy support. I found people that will program it in for about $500 bucks. I have a post in the discussions tab to see if anybody wants to throw in for it to be built. Ideally, a few people can jump in so just one person doesnt have to pay it all.

In this comment under an unrelated issue you said that you tried a VPN and you still encountered the redgifs error. I do have a VPN, and this application uses it correctly.

Under the hood, we use the python requests library to make requests. Requests should use a VPN if you have it correctly configured for your system. Are you sure you have it configured properly? If so, are you sure your redgifs ban is related to this application?

Jmanfrom87 commented 4 months ago

@MalloyDelacroix

When you turn on the VPN the Gifs load in Redgifs. However, when using the scraper with the VPN it only works for a little while before Redgifs bans it as well. No question that the ban is happening because of the app.

magdev2022 commented 4 months ago

to avoid banned, proxy rotation is required.

Jmanfrom87 commented 4 months ago

Yes, that is what I am advocating for.

I am just having it built myself. If anybody needs it they can contact me.

Anonymouse6661 commented 3 months ago

Any luck in developing a solution for this issue?

Jmanfrom87 commented 2 months ago

I had proxy support added but it really doesnt work all that great. Cost me $500 bucks and while it works, it leaves a lot to be desired