Jackhammer9 / RedDownloader

A python library to simplify downloading different types of media from reddit
GNU General Public License v3.0
79 stars 14 forks source link

Stalls on "Fetching Posts..." #26

Closed SkriptKiddy42069 closed 1 year ago

SkriptKiddy42069 commented 1 year ago

Hello,

Occasionally, when I go to run my test code utilizing RedDownloader, it would stall on "Fetching Posts...". If I let it sit long enough, it will eventually produce an error.

Here is my example code: """ from RedDownloader import RedDownloader as rd

rd.DownloadVideosBySubreddit("TikTokCringe", 5, SortBy="top", quality=1080, output='TikTokCringeVideos', destination=r'[insert path here]') """

Error: Fetching Posts... Unable to fetch posts Expecting value: line 1 column 1 (char 0)

I can access the internet on my PC, my other code works just fine, etc.

Please advise.

Thank you!

Jackhammer9 commented 1 year ago

The Issue

This issue is a well known issue faced by a lot of people, there is nothing wrong with your internet connectivity the issue lies in the web service architecture that handles the requests you send. RedDownloader was never designed to be able to handle a large number of requests but as of today it is almost nearing 4 Million requests handled with this great of a traffic comes difficulty for the web server (which is pretty weak itself with just one worker to handle all requests on a free tier hosting website) to adequately divide resources. So anytime you face this error is due to fact that either the webserver is 100% busy or a lot of people are sending huge number of download requests at the same time.

What Am I Doing To Tackle It

I am investing a lot of time to make a stronger web app architecture that can fulfill RedDownloader's Huge Demands it may take some time as I am just a college student bound by Financial and Time Constraints. There is an idea to release the open source version of the web app so anyone can set it up locally to download posts from reddit but again that defeats the whole purpose of Download medias in a single line

What you can do?

while facing an issue like this can be really frustrating sometimes just trying in a bit work just fine as RedDownloader frees up space on the main web server.


Closing the issue as it is unsolvable at the moment, please reopen it if required so