search

conveyal / otpa-cluster

Cluster backend for otpa many-to-many queries.
0 stars 1 forks source link

Set chunk size based on moving average of response times #21

Closed mattwigway closed 9 years ago

mattwigway commented 9 years ago

Keep track of the response times for the last, say, 100 requests, and set the chunk size based on that.

mattwigway commented 9 years ago

Actually we don't need to do that. We expand the chunk size each time there is an underrun. We can make it smaller each time the worker is polled and there are more than chunksize requests remaining.