During rollout in staging, we realized that due to a yet unknown bug the eds cache had drifted causing UH errors in many critical services. We need a way for the oncall to reset the cache when this happens so that the issue can be quickly mitigated.
The reset should be based on the cache key . We should also create a way to globally reset all cache. This will help quickly reset cache without restarting the server in case of a cache drift.
jyotimahapatra
commented
4 years ago
During rollout in staging, we realized that due to a yet unknown bug the eds cache had drifted causing UH errors in many critical services. We need a way for the oncall to reset the cache when this happens so that the issue can be quickly mitigated.
The reset should be based on the cache key . We should also create a way to globally reset all cache. This will help quickly reset cache without restarting the server in case of a cache drift.