Is your feature request related to a problem? Please describe.
In case requesting a feed fails, e.g. due to network issues or exceeded quotas, lamassu currently continously tries to request the feed again, causing a huge load on the service, even though quotas are already exceeded.
Goal / high level use-case
Avoid useless requests, i.e. when a request quota is already exceeded
Describe the solution you'd like
Lamassu should use an exponentially decay when re-requestig this feed again.
Describe alternatives you've considered
In our setup, we already patch ttl values of feeds before importing them to lamassu, to reflect their providers intended request quotas. This proxy could implement a caching/delaying strategy. Nevertheless, I think this should be supported by lamassu.
Additional context
As suggested by @testower in a private conversation, gbfs-loader-java's GBFSFeedUpdater would be the location where such a strategy could be implementend. I'll provide a PR where GBFSFeedUpdater (i.e. it's UpdateStrategy) counts the number of failed request attempts in a row and backs off exponentially. I'll provide a PR for a suggested change.
testower
commented
8 months ago
Is your feature request related to a problem? Please describe. In case requesting a feed fails, e.g. due to network issues or exceeded quotas, lamassu currently continously tries to request the feed again, causing a huge load on the service, even though quotas are already exceeded.
Goal / high level use-case Avoid useless requests, i.e. when a request quota is already exceeded
Describe the solution you'd like Lamassu should use an exponentially decay when re-requestig this feed again.
Describe alternatives you've considered In our setup, we already patch ttl values of feeds before importing them to lamassu, to reflect their providers intended request quotas. This proxy could implement a caching/delaying strategy. Nevertheless, I think this should be supported by lamassu.
Additional context As suggested by @testower in a private conversation,
gbfs-loader-java's GBFSFeedUpdater would be the location where such a strategy could be implementend. I'll provide a PR where GBFSFeedUpdater (i.e. it's UpdateStrategy) counts the number of failed request attempts in a row and backs off exponentially. I'll provide a PR for a suggested change.