Esri / resource-proxy

Proxy files for DotNet, Java and PHP.
Apache License 2.0
371 stars 322 forks source link

Q. Is rateLimitPeriod specified in seconds or minutes? #383

Closed bsvensson closed 8 years ago

bsvensson commented 8 years ago

https://github.com/Esri/resource-proxy/blob/master/README.md says time period (in minutes).

However https://github.com/Esri/resource-proxy/blob/master/PHP/proxy.php#L2001 says $this->rate = $ratelimit / $ratelimitperiod / 60; //ratelimitperiod is designed to be in seconds

While https://github.com/Esri/resource-proxy/blob/master/DotNet/proxy.ashx#L203 has this snippet: RateLimitPeriod + " minute(s). ...

cc: @MikeTschudi

esoekianto commented 8 years ago

nice catch!

@bsvensson should we change the doc to say in seconds or change the code to be in minute?

MikeTschudi commented 8 years ago

.Net version also has conversion _rate = (double) rate_limit / rate_limit_period / 60;

Operator precedence in .Net, PHP, and Java handles this as ((rate_limit / rate_limit_period) / 60).

((rate_limit / rate_limit_period) / 60) is the same as (rate_limit / rate_limit_period) * (1 / 60), or (requests / minute) * (minutes / second) = requests / second; thus _rate is in requests per second.

Other usage in the .Net code confirms that rateLimitPeriod is in minutes; here's how the internal count for testing against the rate limit cap is calculated:

 TimeSpan ts = DateTime.Now - _lastUpdate;
_lastUpdate = DateTime.Now;
//assuming uniform distribution of requests over time,
//reducing the counter according to # of seconds passed
//since last invocation
_count = Math.Max(0, _count - ts.TotalSeconds * _rate);

Seconds elapsed are multiplied by requests/second to get a count of request "credits" to apply.

Empirical evidence with the .Net flavor supports the minutes interpretation. I set the limit to 6 requests per minute

rateLimit="6"
rateLimitPeriod="1">

and added logging to show the net value of the internal rate count with each request. If the rate limit period had been seconds, I would not have been able to hit the rate limit—just couldn’t test that fast—but I did hit the limit:

2016-09-08 13:04:25  Pair [http://geoenrich.arcgis.com/arcgis/rest/]x[10.29.77.63] was rate checked. Count 1/6
2016-09-08 13:04:28  Pair [http://geoenrich.arcgis.com/arcgis/rest/]x[10.29.77.63] was rate checked. Count 1.62116329/6
2016-09-08 13:04:40  Pair [http://geoenrich.arcgis.com/arcgis/rest/]x[10.29.77.63] was rate checked. Count 1.44875572/6
2016-09-08 13:04:43  Pair [http://geoenrich.arcgis.com/arcgis/rest/]x[10.29.77.63] was rate checked. Count 2.20233238/6
2016-09-08 13:04:45  Pair [http://geoenrich.arcgis.com/arcgis/rest/]x[10.29.77.63] was rate checked. Count 3.00191345/6
2016-09-08 13:04:48  Pair [http://geoenrich.arcgis.com/arcgis/rest/]x[10.29.77.63] was rate checked. Count 3.67328207/6
2016-09-08 13:04:50  Pair [http://geoenrich.arcgis.com/arcgis/rest/]x[10.29.77.63] was rate checked. Count 4.42725899/6
2016-09-08 13:04:53  Pair [http://geoenrich.arcgis.com/arcgis/rest/]x[10.29.77.63] was rate checked. Count 5.12172986/6
2016-09-08 13:04:56  Pair [http://geoenrich.arcgis.com/arcgis/rest/]x[10.29.77.63] was rate checked. Count 5.84910392/6
2016-09-08 13:05:32  Pair [http://geoenrich.arcgis.com/arcgis/rest/]x[10.29.77.63] was rate checked. Count 3.24236876/6
2016-09-08 13:05:36  Pair [http://geoenrich.arcgis.com/arcgis/rest/]x[10.29.77.63] was rate checked. Count 3.89413624/6
2016-09-08 13:05:38  Pair [http://geoenrich.arcgis.com/arcgis/rest/]x[10.29.77.63] was rate checked. Count 4.69801748/6
2016-09-08 13:05:40  Pair [http://geoenrich.arcgis.com/arcgis/rest/]x[10.29.77.63] was rate checked. Count 5.4954982/6
2016-09-08 13:05:42  Pair [http://geoenrich.arcgis.com/arcgis/rest/]x[10.29.77.63] was rate checked. Count 6.23817337/6
2016-09-08 13:05:46  Pair [http://geoenrich.arcgis.com/arcgis/rest/]x[10.29.77.63] was rate checked. Count 6.87953973/6
2016-09-08 13:05:48  Pair [http://geoenrich.arcgis.com/arcgis/rest/]x[10.29.77.63] is throttled to 6 requests per 1 minute(s). Come back later.

So it appears to me that the only error is the one that @lindsayking found on the https://developers.arcgis.com/authentication/working-with-proxies/#proxy-service-configuration page: "On a service by service basis configure a rateLimit and rateLimitPeriod (in seconds) to limit request frequency during the rate limit period.", and I submitted it via that page's feedback link.

(And the comment in the PHP code--I'd forgotten that one.)

bsvensson commented 8 years ago

Great - thanks for digging into that @MikeTschudi.

I'll update the PHP proxy comment and https://developers.arcgis.com/authentication/working-with-proxies/#proxy-service-configuration.

bsvensson commented 8 years ago

Doc updated. Closing...