search

foyer-rs / foyer

Hybrid in-memory and disk cache in Rust
https://foyer.rs
Apache License 2.0
291 stars 20 forks source link

refactor: revert the pre-serialization and parallel buffer #717

Closed MrCroxx closed 1 month ago

MrCroxx commented 1 month ago

What's changed and what's your intention?

Please explain IN DETAIL what the changes are in this PR and why they are needed. :D

Let's revert the pre-serialization and parallel buffer design to resolve the memory usage overhead.

Checklist

Related issues or PRs (optional)

close #708

MrCroxx commented 1 month ago

Benchmark

command

cargo build --release --features jemalloc && PATH=/home/mrcroxx/.local/bin:$PATH RUST_BACKTRACE=1 RUST_LOG=info flamegraph -- ./target/release/foyer-bench --file /dev/nvme2n1 --mem 64MiB --disk 100GiB --region-size 64MiB --get-range 100000 --flushers 4 --reclaimers 4 --time 60 --writers 256 --w-rate 4MiB --admission-rate-limit 500MiB --readers 32 --r-rate 32MiB --metrics --warm-up 2 --runtime separated --user-runtime-worker-threads 8 --read-runtime-worker-threads 8 --write-runtime-worker-threads 4 --read-runtime-max-blocking-threads 8 --write-runtime-max-blocking-threads 4

main

Total:
disk total iops: 16007.2
disk total throughput: 1.5 GiB/s
disk read iops: 14971.7
disk read throughput: 1.0 GiB/s
disk write iops: 1035.5
disk write throughput: 498.8 MiB/s
insert iops: 15859.0/s
insert throughput: 1.0 GiB/s
insert lat p50: 4us
insert lat p90: 8us
insert lat p99: 14us
insert lat p999: 28us
insert lat p9999: 68us
insert lat p99999: 214us
insert lat pmax: 475us
get iops: 45936.5/s
get miss: 65.49% 
get throughput: 1.0 GiB/s
get hit lat p50: 299us
get hit lat p90: 627us
get hit lat p99: 2239us
get hit lat p999: 2815us
get hit lat p9999: 3167us
get hit lat p99999: 3519us
get hit lat pmax: 4063us
get miss lat p50: 15us
get miss lat p90: 30us
get miss lat p99: 68us
get miss lat p999: 213us
get miss lat p9999: 309us
get miss lat p99999: 461us
get miss lat pmax: 639us

PR

Total:
disk total iops: 22340.9
disk total throughput: 1.4 GiB/s
disk read iops: 14871.8
disk read throughput: 1.0 GiB/s
disk write iops: 7469.0
disk write throughput: 496.0 MiB/s
insert iops: 15858.9/s
insert throughput: 1.0 GiB/s
insert lat p50: 3us
insert lat p90: 6us
insert lat p99: 11us
insert lat p999: 23us
insert lat p9999: 59us
insert lat p99999: 263us
insert lat pmax: 507us
get iops: 49758.2/s
get miss: 68.14% 
get throughput: 1.0 GiB/s
get hit lat p50: 291us
get hit lat p90: 631us
get hit lat p99: 2191us
get hit lat p999: 2751us
get hit lat p9999: 3279us
get hit lat p99999: 5663us
get hit lat pmax: 6367us
get miss lat p50: 14us
get miss lat p90: 28us
get miss lat p99: 62us
get miss lat p999: 150us
get miss lat p9999: 303us
get miss lat p99999: 503us
get miss lat pmax: 1191us
codecov[bot] commented 1 month ago

Codecov Report

Attention: Patch coverage is 75.70621% with 43 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
foyer-storage/src/large/flusher.rs 68.75% 20 Missing :warning:
foyer-storage/src/engine.rs 18.18% 9 Missing :warning:
foyer-storage/src/serde.rs 75.00% 6 Missing :warning:
foyer-storage/src/storage/either.rs 0.00% 4 Missing :warning:
foyer-storage/src/large/batch.rs 95.23% 3 Missing :warning:
foyer-storage/src/small/generic.rs 0.00% 1 Missing :warning:
Files with missing lines Coverage Δ
foyer-storage/src/large/generic.rs 87.77% <100.00%> (-0.30%) :arrow_down:
foyer-storage/src/storage/noop.rs 73.91% <100.00%> (-3.45%) :arrow_down:
foyer-storage/src/store.rs 73.12% <100.00%> (-0.51%) :arrow_down:
foyer-storage/src/small/generic.rs 0.00% <0.00%> (ø)
foyer-storage/src/large/batch.rs 83.20% <95.23%> (-0.57%) :arrow_down:
foyer-storage/src/storage/either.rs 0.00% <0.00%> (ø)
foyer-storage/src/serde.rs 84.21% <75.00%> (-12.22%) :arrow_down:
foyer-storage/src/engine.rs 34.45% <18.18%> (-1.20%) :arrow_down:
foyer-storage/src/large/flusher.rs 85.14% <68.75%> (-10.18%) :arrow_down:

... and 4 files with indirect coverage changes

hzxa21 commented 1 month ago

The disk write iops increases from 1035.5 to 7469.0 with similar disk throughput and insert iops. Is this expected?

MrCroxx commented 1 month ago

The disk write iops increases from 1035.5 to 7469.0 with similar disk throughput and insert iops. Is this expected?

Yes. The disk hit ratio also increased. I'm still checking the reason.

MrCroxx commented 1 month ago

Discussed with @st1page offline. Let's merge it.