karlseguin / ccache

A golang LRU Cache for high concurrency
MIT License
1.27k stars 118 forks source link

High lock contention in LayeredCache.set with few primary keys #73

Closed bep closed 2 years ago

bep commented 2 years ago

I understand that the problem I'm describing isn't a problem for LayeredCache's intended use (as described in readme).

But I had an idea that I could use ccache as

But with the above I would only have a handful of primary keys, so each partition will end up in its own bucket, and I, not surprisingly, see lots of lock contention in the profiler.

I can certainly simplify my setup to not use the LayeredCache, but it would be convenient, and In my head this should be fixed if the bucket method below considered both keys in the hash:

func (c *LayeredCache) set(primary, secondary string, value interface{}, duration time.Duration, track bool) *Item {
    item, existing := c.bucket(primary).set(primary, secondary, value, duration, track)
    if existing != nil {
        c.deletables <- existing
    }
    c.promote(item)
    return item
}

func (c *LayeredCache) bucket(key string) *layeredBucket {
    h := fnv.New32a()
    h.Write([]byte(key))
    return c.buckets[h.Sum32()&c.bucketMask]
}
bep commented 2 years ago

Never mind, the above obviously would not work for DeleteFunc and others needing the same bucket...