cloudflare / pingora

A library for building fast, reliable and evolvable network services.
Apache License 2.0
20.25k stars 1.1k forks source link

Updated flurry (Cache Performance Improvement) #248

Open NathanielPrescott opened 1 month ago

NathanielPrescott commented 1 month ago

Since Pingora became open sourced I've been experimenting and learning as much as I can, especially from TinyUFO. While messing with TinyUFO I found a comment about not using Rust version 1.71 even though the MSVR is 1.72. I've upgraded the flurry and quick_cache versions and then ran the TinyUFO benches. The results are run using v0.5.1 for each crate, on a arch linux system with an 8 core CPU. I'm not sure how this change would look on a more relevant server setup, but I would be interested in seeing the results.

The bench_hit_ratio and bench_memory look the same or similar enough between both version.

bench_perf (Ops per Second): Cache Type Type 0.4.0 0.5.1 diff
lru single read 27215914 29318238 7.72%
moka single read 10167350 9408540 -7.46%
quick_cache single read 33161216 35265832 6.35%
tinyufo single read 20067244 22591294 12.58%
tinyufo compact single read 13532442 14353295 6.07%
lru concurrent read 5536601 6433813 16.21%
moka concurrent read 12408710 12613696 1.65%
quick_cache concurrent read 23902774 44858196 87.67%
tinyufo concurrent read 144783008 173088224 19.55%
tinyufo compact concurrent read 69050896 75399064 9.19%
lru mixed read/write 5445493 5493046 0.87%
moka mixed read/write 13652672 13983480 2.42%
quick_cache mixed read/write 54678828 91533376 67.40%
tinyufo mixed read/write 54712464 60108592 9.86%
tinyufo compact mixed read/write 55222964 59824780 8.33%

As you can see there is a significant increase in performance for TinyUFO, though quick_cache has an absurd increase. Let me know if you have any questions or concerns with updating these crates.

NathanielPrescott commented 1 week ago

Any updates on this one?