Benchmarking Cache and WorkerPool implementations

thrawn01 commented 3 months ago

Purpose

Trying to diagnose some performance issues we notice in production during high request volume. The theory is that we are running into mutex contention. As a result we wrote WorkerPool to partition the internal cache to reduce contention. However, we never benchmarked contention. This PR is intended to add benchmarking and compare different implementations of WorkerPool.

Implementation

Added WorkerMutex which is just an LRU cache with a mutex around it
Added WorkerNoCache which is like WorkerMutex but the cache doesn't actually do anything. It's intended to be a base line to compare the other implementations with.
Added WorkerOtter which uses an high performance golang cache called https://maypok86.github.io/otter/
Added AccessStructure tests to figure out how to graph the results.
Added lockfree to implement a lock free version of the cache based on some papers I found. I haven't finished it as I got distracted by strange benchmark results. 😄

I created a golang application to graph the results of the benchmark. https://github.com/gubernator-io/gubernator-graphs

See benchmark_plot.html in the graphs repo for the latest results.

Status

I think I need to run these benchmarks on a machine with more than 10 Cores (My Laptop). I'll work on doing that this ~weekend or next week~ sometime in the future.

thrawn01 commented 2 months ago

Benchmark running on a 32 core machine. Otter Performance Benchmark on 32 core machine

thrawn01 commented 1 month ago

The result of this work has been rolled into #15

gubernator-io / gubernator