CoreyKaylor / Lightning.NET

.NET library for LMDB key-value store
Other
399 stars 82 forks source link

Add a basic write benchmark #117

Closed AlgorithmsAreCool closed 4 years ago

AlgorithmsAreCool commented 4 years ago

This PR closes #114

I've added a working Benchmark.Net project and some very basic write performance tests.

On my machine they take several minutes to run and produced the following


BenchmarkDotNet=v0.12.1, OS=Windows 10.0.18363.836 (1909/November2018Update/19H2)
Intel Core i7-8750H CPU 2.20GHz (Coffee Lake), 1 CPU, 12 logical and 6 physical cores
.NET Core SDK=5.0.100-preview.5.20279.10
  [Host]     : .NET Core 3.1.5 (CoreCLR 4.700.20.26901, CoreFX 4.700.20.27001), X64 RyuJIT
  DefaultJob : .NET Core 3.1.5 (CoreCLR 4.700.20.26901, CoreFX 4.700.20.27001), X64 RyuJIT
Method BatchSize WriteSize SequentialKeys Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated
Write 1 4 True 1.628 μs 0.0220 μs 0.0206 μs 0.0629 - - 304 B
Write 1 8 True 1.605 μs 0.0101 μs 0.0079 μs 0.0629 - - 304 B
Write 1 64 True 1.602 μs 0.0075 μs 0.0058 μs 0.0629 - - 304 B
Write 1 256 True 1.620 μs 0.0210 μs 0.0196 μs 0.0629 - - 304 B
Write 10 4 True 2.590 μs 0.0098 μs 0.0082 μs 0.0610 - - 304 B
Write 10 8 True 2.620 μs 0.0333 μs 0.0311 μs 0.0610 - - 304 B
Write 10 64 True 2.605 μs 0.0264 μs 0.0220 μs 0.0610 - - 304 B
Write 10 256 True 2.703 μs 0.0282 μs 0.0264 μs 0.0610 - - 304 B
Write 100 4 True 14.852 μs 0.2910 μs 0.5322 μs 0.0610 - - 304 B
Write 100 8 True 14.730 μs 0.2944 μs 0.7386 μs 0.0610 - - 304 B
Write 100 64 True 15.623 μs 0.3049 μs 0.4373 μs 0.0610 - - 304 B
Write 100 256 True 21.123 μs 0.4208 μs 0.5471 μs 0.0610 - - 304 B
CoreyKaylor commented 4 years ago

This looks good, I can pull in and merge into a pre-Span branch if you'd like. Obviously we can add more over time but this is a good start.

AlgorithmsAreCool commented 4 years ago

I'll leave that to your discretion, but this at least helps give a baseline to make perf changes against.

CoreyKaylor commented 4 years ago

From my mac on the pre-span branch.

Method BatchSize WriteSize SequentialKeys Mean Error StdDev Median Gen 0 Gen 1 Gen 2 Allocated
Write 1 4 True 1.745 μs 0.0194 μs 0.0182 μs 1.738 μs 0.0458 - - 384 B
Write 1 8 True 1.763 μs 0.0132 μs 0.0110 μs 1.761 μs 0.0458 - - 384 B
Write 1 64 True 1.843 μs 0.0249 μs 0.0221 μs 1.849 μs 0.0458 - - 384 B
Write 1 256 True 1.888 μs 0.0376 μs 0.0573 μs 1.910 μs 0.0458 - - 384 B
Write 10 4 True 4.759 μs 0.0949 μs 0.2365 μs 4.899 μs 0.1297 - - 1104 B
Write 10 8 True 4.313 μs 0.0862 μs 0.2031 μs 4.419 μs 0.1297 - - 1104 B
Write 10 64 True 4.332 μs 0.0861 μs 0.2145 μs 4.478 μs 0.1297 - - 1104 B
Write 10 256 True 4.422 μs 0.0884 μs 0.2421 μs 4.553 μs 0.1297 - - 1104 B
Write 100 4 True 30.561 μs 0.6006 μs 1.2928 μs 30.878 μs 0.9766 - - 8304 B
Write 100 8 True 30.082 μs 0.5921 μs 1.3840 μs 29.725 μs 0.9766 - - 8304 B
Write 100 64 True 32.152 μs 0.6380 μs 1.5410 μs 31.529 μs 0.9766 - - 8304 B
Write 100 256 True 36.147 μs 0.7208 μs 1.7269 μs 36.146 μs 0.9766 - - 8304 B

On the master branch

Method BatchSize WriteSize SequentialKeys Mean Error StdDev Median Gen 0 Gen 1 Gen 2 Allocated
Write 1 4 True 1.154 μs 0.0190 μs 0.0233 μs 1.148 μs 0.0362 - - 304 B
Write 1 8 True 1.146 μs 0.0061 μs 0.0048 μs 1.145 μs 0.0362 - - 304 B
Write 1 64 True 1.159 μs 0.0095 μs 0.0084 μs 1.158 μs 0.0362 - - 304 B
Write 1 256 True 1.202 μs 0.0238 μs 0.0377 μs 1.194 μs 0.0362 - - 304 B
Write 10 4 True 2.295 μs 0.0446 μs 0.0596 μs 2.292 μs 0.0343 - - 304 B
Write 10 8 True 2.131 μs 0.0426 μs 0.1084 μs 2.111 μs 0.0343 - - 304 B
Write 10 64 True 2.400 μs 0.0478 μs 0.1217 μs 2.478 μs 0.0343 - - 304 B
Write 10 256 True 2.078 μs 0.0351 μs 0.0625 μs 2.055 μs 0.0343 - - 304 B
Write 100 4 True 12.806 μs 0.2391 μs 0.2559 μs 12.724 μs 0.0305 - - 304 B
Write 100 8 True 12.690 μs 0.2414 μs 0.2371 μs 12.713 μs 0.0305 - - 304 B
Write 100 64 True 14.839 μs 0.2049 μs 0.1917 μs 14.839 μs 0.0305 - - 304 B
Write 100 256 True 18.726 μs 0.3654 μs 0.3589 μs 18.770 μs 0.0305 - - 304 B
AlgorithmsAreCool commented 4 years ago

@CoreyKaylor So i had a bug in the benchmark, I forgot to commit the transaction 😅

But this does show API and P/Invoke overhead nicely