search

allegro / bigcache

Efficient cache for gigabytes of data written in Go.
http://allegro.tech/2016/03/writing-fast-cache-service-in-go.html
Apache License 2.0
7.45k stars 593 forks source link

Why doesn't bigcache consider designing expiration time for each cache key? #360

Open zsyu9779 opened 1 year ago

zsyu9779 commented 1 year ago

During the process of using and reading the source code, it was found that in each instance of bigcache, all key-value pairs can only be set with a unified expiration time. A time.Ticker is used to periodically check if the cache has expired based on the interval set by CleanWindow. So why wasn't a method considered to allow setting expiration time for each cache key?

janisz commented 1 year ago

So why wasn't a method considered to allow setting expiration time for each cache key?

While it is technically feasible to implement such a feature, it could introduce complexity to the cache system. The bigcache library is based on a circular buffer (i.e., queue.BytesQueue) and does not have a defragmentation process. If we allow every key to be removed at different times, it could result in a significant amount of unused memory. On the other hand, we do allow the removal of a single key, but it is more like marking it as obsolete rather than actually deleting it.

zsyu9779 commented 1 year ago

So why wasn't a method considered to allow setting expiration time for each cache key?

While it is technically feasible to implement such a feature, it could introduce complexity to the cache system. The bigcache library is based on a circular buffer (i.e., queue.BytesQueue) and does not have a defragmentation process. If we allow every key to be removed at different times, it could result in a significant amount of unused memory. On the other hand, we do allow the removal of a single key, but it is more like marking it as obsolete rather than actually deleting it.

Regarding the problem of memory waste, can we consider designing a compression algorithm? That is, when operating the bytesQueue, record the number of expired keys, and when the number of expired keys reaches a certain threshold, start a compression algorithm to asynchronously create a new, expanded bytesQueue. Then move the data from the old bytesQueue to the new one. During this process, read and write requests are synchronized between the two bytesQueues.

zsyu9779 commented 1 year ago

So why wasn't a method considered to allow setting expiration time for each cache key?

While it is technically feasible to implement such a feature, it could introduce complexity to the cache system. The bigcache library is based on a circular buffer (i.e., queue.BytesQueue) and does not have a defragmentation process. If we allow every key to be removed at different times, it could result in a significant amount of unused memory. On the other hand, we do allow the removal of a single key, but it is more like marking it as obsolete rather than actually deleting it.

Regarding the problem of memory waste, can we consider designing a compression algorithm? That is, when operating the bytesQueue, record the number of expired keys, and when the number of expired keys reaches a certain threshold, start a compression algorithm to asynchronously create a new, expanded bytesQueue. Then move the data from the old bytesQueue to the new one. During this process, read and write requests are synchronized between the two bytesQueues.

However, this approach does seem to introduce some complexity to the entire system :(

janisz commented 1 year ago

In my opinion, we need to carefully consider the trade-off between complexity and usability when evaluating changes to our system. While adding new features can improve functionality, it can also introduce greater maintenance complexity. It's worth exploring other solutions available in the market for in-memory caching, as they may offer better performance or better align with specific requirements. For instance, the Bigcache solution is particularly well-suited for static maps that are not frequently modified after the application startup.