search

szyb / synopackage_dotnet

This is source code of the page https://synopackage.com - a search engine for third-party packages dedicated to Synology's DSM system. Also act as proxy repository: one link -> all sources. Written in C# & Typescript.
https://synopackage.com
Apache License 2.0
17 stars 3 forks source link

Better handling with inactive sources #312

Open szyb opened 2 years ago

szyb commented 2 years ago

Short description Sometimes sources not working for a while and then comeback alive. During the downtime, when our cache is expired, each request from WWW and DSM are sent directly to the source server, so it can flood the source server with our requests. This can lead to block website's IP(s) as it acts as potential DDOS attack.

Motivation

szyb commented 2 years ago

I'm thinking of auto disable feature. It could disable source for a while and check periodically whether enable it or not.

szyb commented 2 years ago

Now I'm thinking about counting number of failures and decide when to query server one more time. i.e, (just a sample): when failureCount < 5 then query the server (no wait) when >=5 and < 10 then schedule next query to be executed in an hour when >=10 and < 20 then schedule next query to be executed in 4 hours when >= 20 and < 30 then schedule next query to be executed in 12 hours when >= 30 then schedule next query to be executed in 24 hours In this sample after ~7 days we will send request only once per 24 hours.

Need to also clear number of failures when valid response will be received. FailureCount should be stored per source (no matter what version or model is used) and maybe kept in memory (so the restart will clear the state)