search

quakkels / rssdiscoveryengine

The RSS Discovery Engine exists to encourage people to use RSS for finding and consuming their news and current events.
MIT License
159 stars 9 forks source link

This app is making a large amount of requests in a short amount of time #30

Open markcellus opened 1 year ago

markcellus commented 1 year ago

Hey, this project appears to continuously make a large amount of requests to our server in a very short amount of time. In just one of the incidents, we see that the app has made over 85 requests to the same URL on our servers within 600 milliseconds.

We've decided to temporarily block this app from accessing it for now. But I definitely want to see if there's a remedy here because we like what you're doing with this project.

I haven't taken a deep look at the code here, but do you have some mechanisms in place to prevent so many requests from happening at the same time?

Thanks for any information.

quakkels commented 2 months ago

Sorry for the long overdue response, and for the flood of requests. The site has been taken down for now. However this project is not abandoned. I will be working on the next version soon, and it will include request rate limiting, and also likely some caching.

I'm also thinking about how I may be able to deter bots from abusing a service like this without hindering real users, and also respect site owner's feeds, servers, processing, and bandwidth, etc.

I'm passionate about having an independent internet. That means not relying on large providers like Google and CloudFlare.

Apologies for the inconvernience of RSS Discovery Engine, and also apologies for the very late response.