At the moment, every time there's a request for the predictor, there's a hit on the model store to check for updated models.
In a high traffic scenario this can lead to a degradation of the predictor's responsiveness.
In this PR a configurable time-between-checks in milliseconds is introduced via the env var MODEL_STORE_CHECK_RATE (e.g. a value of 60000 will check for updated models every minute).
The previous behaviour (a check with every request) can be simulated by setting the interval to 0.
At the moment, every time there's a request for the predictor, there's a hit on the model store to check for updated models.
In a high traffic scenario this can lead to a degradation of the predictor's responsiveness.
In this PR a configurable time-between-checks in milliseconds is introduced via the env var
MODEL_STORE_CHECK_RATE
(e.g. a value of60000
will check for updated models every minute).The previous behaviour (a check with every request) can be simulated by setting the interval to
0
.@elmiko @zmhassan @sophwats ptal.