Currently, we have two cache types(Block and Config) that uses lock mechanism to avoid parallel cache generation and excessive resource utilization.
So every time we want to generate and write a new cache, we acquire a lock and parallel process wait until lock released.
When the lock is released, customer get fresh data from storage.
We usually think that trade-off with lock waiting is acceptable from the performance side.
But the larger amount of Blocks or Cache merchant has, the more time he will wait in locks.
In some scenarios, we could wait numbers of keys * lookup timeout amount of time in parallel process.
We noticed that in some rare cases merchant can have hundreds keys in Block/Config cache, so even small lookup timeout for lock may cost seconds.
Solution
Was proposed by IvanChepurnyi in his PR
Instead of wait until new data is written we will send stale cache while new one is generated, so there will be no locks.
Problem
Currently, we have two cache types(Block and Config) that uses lock mechanism to avoid parallel cache generation and excessive resource utilization. So every time we want to generate and write a new cache, we acquire a lock and parallel process wait until lock released. When the lock is released, customer get fresh data from storage.
We usually think that trade-off with lock waiting is acceptable from the performance side. But the larger amount of Blocks or Cache merchant has, the more time he will wait in locks. In some scenarios, we could wait numbers of keys * lookup timeout amount of time in parallel process. We noticed that in some rare cases merchant can have hundreds keys in Block/Config cache, so even small lookup timeout for lock may cost seconds.
Solution
Was proposed by IvanChepurnyi in his PR Instead of wait until new data is written we will send stale cache while new one is generated, so there will be no locks.
Requested Reviewers
@kandy