intelligenthack / intelligentcache

A distributed cache backed by Redis

MIT License

18 stars 2 forks source link

3.1 release PR #33

Closed sklivvz closed 2 years ago

sklivvz commented 3 years ago

Changes

Major

RedisCache now will calculate the value only once reliably under heavy loads
MemoryCache has a much more efficient locking strategy (locks are reader-writer and are now per key instead of per cache)

Minor

MemoryCache can use a custom MemoryCache if needed, just pass it in the constructor
ProtobufSerializer will now throw an exception if it does not know how to handle a Compression

Performance Improvements

Optimistic case

In this benchmark, the caches are always hit.

Before

Method	Mean	Error	StdDev	Gen 0	Gen 1	Gen 2	Allocated
CompositeMultiThread	153.9 μs	1.45 μs	1.28 μs	51.2695	5.8594	-	281 KB
CompositeSingleThread	282.3 μs	1.23 μs	1.03 μs	42.9688	0.9766	-	243 KB
MemoryMultiThread	141.3 μs	1.15 μs	1.07 μs	31.9824	3.1738	-	171 KB
MemorySingleThread	248.3 μs	1.59 μs	1.41 μs	23.4375	0.4883	-	133 KB
RedisMultiThread	68,087.1 μs	1,174.50 μs	1,098.63 μs	625.0000	-	-	3,141 KB
RedisSingleThread	790,697.3 μs	12,142.78 μs	11,358.36 μs	-	-	-	3,103 KB

After

Method	Mean	Error	StdDev	Gen 0	Gen 1	Gen 2	Allocated
CompositeMultiThread	983.8 μs	107.03 μs	315.58 μs	21.4844	1.9531	-	117 KB
CompositeSingleThread	127.9 μs	1.38 μs	1.29 μs	13.1836	0.2441	-	75 KB
MemoryMultiThread	1,135.5 μs	127.58 μs	376.16 μs	15.6250	-	-	83 KB
MemorySingleThread	113.8 μs	0.59 μs	0.49 μs	7.3242	-	-	41 KB
RedisMultiThread	20,710.3 μs	411.92 μs	1,010.45 μs	218.7500	31.2500	-	976 KB
RedisSingleThread	241,469.1 μs	3,196.43 μs	2,989.95 μs	-	-	-	932 KB

Pessimistic case

In this benchmark, the caches are always missed.

Before

Method	Mean	Error	StdDev	Gen 0	Gen 1	Gen 2	Allocated
CompositeMultiThread	4,179.6 ms	80.30 ms	95.59 ms	-	-	-	745 KB
CompositeSingleThread	4,171.1 ms	82.78 ms	73.39 ms	-	-	-	774 KB
MemoryMultiThread	4,259.5 ms	84.84 ms	83.33 ms	-	-	-	260 KB
MemorySingleThread	4,232.1 ms	83.80 ms	89.67 ms	-	-	-	187 KB
RedisMultiThread	292.3 ms	5.70 ms	6.34 ms	-	-	-	554 KB
RedisSingleThread	4,203.8 ms	84.02 ms	78.59 ms	-	-	-	510 KB

After

Method	Mean	Error	StdDev	Gen 0	Gen 1	Gen 2	Allocated
CompositeMultiThread	291.2 ms	5.01 ms	6.52 ms	-	-	-	1,172 KB
CompositeSingleThread	4,179.8 ms	75.79 ms	70.89 ms	-	-	-	937 KB
MemoryMultiThread	296.6 ms	5.45 ms	8.16 ms	-	-	-	339 KB
MemorySingleThread	4,263.9 ms	77.93 ms	72.89 ms	-	-	-	387 KB
RedisMultiThread	295.2 ms	5.80 ms	6.91 ms	-	-	-	695 KB
RedisSingleThread	4,200.7 ms	75.37 ms	70.50 ms	-	-	-	652 KB

ocoster-is commented 3 years ago

The only concern I have with this is that the dictionary in MultiKeyLock is unbounded and there is no way to evict keys from it. Over time this dictionary could become a bottleneck if it gains a lot of entries (and can become a memory hog, as each ReaderWriterLockSlim takes > 40 bytes).

Mentioning this as this class is used in long lived applications (the caches).