search

bbc / alephant-broker

Brokers requests for alephant components
MIT License
4 stars 3 forks source link

[SPIKE] Stale while revalidate #37

Closed stevenjack closed 8 years ago

stevenjack commented 9 years ago

Problem

Currently the request cycle is governed and reliant on the speed of the request being made to the renderer, this can cause issue when the service is slow or has intermittent issues and can cause the broker to end up hammering the renderer with a number of requests.

Solution

Follows a stale-while-revalidate and stale-while-error model, so we change the cache object's ttl to be much higher such as a week/month then store an object inside the cache which has it's own ttl and update time. This allows us to check this when pulling the item out of the cache, and if it's expired then thread off and do the update. All other subsequent requests still serve out of the cache until the thread has got the new content and updated the cache.

We also have a validating and last_validated property so that once the thread is off re-validating the content we don't have lot's of other threads being created to do that same work.

Also the last_validated variable allows the operation to timeout, as the thread may have died or not completed for some reason so we don't want the content to remain stale there after.

bbc-fossbot commented 9 years ago

Does the validating property need to be read consistent? I.e. are multiple servers likely to be making the same validation request and is that likely to cause an issue?

On Fri, 19 Dec 2014 09:50 Steven Jack notifications@github.com wrote:

Problem

Currently the request cycle is governed and reliant on the speed of the request being made to the renderer, this can cause issue when the service is slow or has intermittent issues and can cause the broker to end up hammering the renderer with a number of requests. Solution

Follow a stale-while-revalidate and stale-while-error model, so we change the cache object's ttl to be much higher such as a week/month then store an object inside in the cache which has it's own ttl and update time. This allows us to check this when pulling the item out of the cache, and if it's expired then thread off and do the update. All other subsequent requests still serve out of the cache until the thread has got the new content and updated the cache.

We also have a validating property that and last_validated so that once the thread is off re-validating the content we don't have lot's of other threads being created to do that same work.

Also the last_validated variable allows the operation to timeout, as the thread may have died or not completed for some reason so we don't want the

content to remain stale there after.

You can merge this Pull Request by running

git pull https://github.com/BBC-News/alephant-broker stale-while-revalidate

Or view, comment on, or merge it at:

https://github.com/BBC-News/alephant-broker/pull/37 Commit Summary

  • Initial work on cache class and load strategy

File Changes

Patch Links:

— Reply to this email directly or view it on GitHub https://github.com/BBC-News/alephant-broker/pull/37.

stevenjack commented 9 years ago

@bbc-fossbot Line 30 in cached_http.rb updates the cache so the validating property is set to true so that if another box with the same requests checks the cache and it's expired it won't validate the content as the property is already set in the cached object, so you shouldn't get a case of multiple servers trying to re-validate the same content?

bbc-fossbot commented 9 years ago

Sweet. If the cache server is holding the validating property, it should be fine.

On Fri, 19 Dec 2014 10:09 Steven Jack notifications@github.com wrote:

@bbc-fossbot https://github.com/bbc-fossbot Line 30 (not sure how to ref files in a pull request) in cached_http.rb updates the cache so the validating property is set to true so that if another box with the same requests checks the cache and it's expired it won't validate the content as the property is already set in the cached object, so you shouldn't get a case of multiple servers trying to re-validate the same content?

— Reply to this email directly or view it on GitHub https://github.com/BBC-News/alephant-broker/pull/37#issuecomment-67619933 .

Integralist commented 9 years ago

:+1: