search

CoreyKaylor / Lightning.NET

.NET library for LMDB key-value store
Other
397 stars 82 forks source link

Making Windows auto resize work with netcore3.1 and net5.0 #138

Closed CoreyKaylor closed 3 years ago

CoreyKaylor commented 3 years ago

There may be more work necessary, but this should be a fairly clean solution for the auto grow capability on Windows.

Fixes #131

AlgorithmsAreCool commented 3 years ago

I don't spot any immediate issues with these changes, the test failure is confusing to me

CoreyKaylor commented 3 years ago

I wanted to see if there was a material impact to using the auto-resize lib. It seems on the surface that when staying within the bounds of it's initial allocated size, there is no material impact. I didn't run a benchmark to test growing beyond that initial size, but I suspect that will be when performance will see a bigger hit.

This benchmark was run on a Parallels VM with 8-cores allocated to the VM, only intended to compare the two native libs.

Auto-Resize

Read


BenchmarkDotNet=v0.12.1, OS=Windows 10.0.18362.1016 (1903/May2019Update/19H1)
Intel Core i9-9880H CPU 2.30GHz, 1 CPU, 8 logical and 8 physical cores
.NET Core SDK=5.0.301
  [Host]     : .NET Core 5.0.7 (CoreCLR 5.0.721.25508, CoreFX 5.0.721.25508), X64 RyuJIT
  DefaultJob : .NET Core 5.0.7 (CoreCLR 5.0.721.25508, CoreFX 5.0.721.25508), X64 RyuJIT
Method OpsPerTransaction ValueSize KeyOrder Mean Error StdDev Median Gen 0 Gen 1 Gen 2 Allocated
Read 1 8 Sequential 385.3 ns 7.47 ns 6.99 ns 386.0 ns 0.0362 - - 304 B
Read 1 64 Sequential 385.3 ns 6.76 ns 9.25 ns 384.4 ns 0.0362 - - 304 B
Read 1 256 Sequential 379.5 ns 7.35 ns 10.54 ns 378.5 ns 0.0362 - - 304 B
Read 100 8 Sequential 9,695.0 ns 181.41 ns 178.17 ns 9,704.5 ns 0.0305 - - 304 B
Read 100 64 Sequential 11,638.3 ns 268.54 ns 770.48 ns 11,378.8 ns 0.0305 - - 304 B
Read 100 256 Sequential 11,465.4 ns 185.62 ns 234.75 ns 11,414.2 ns 0.0305 - - 304 B
Read 1000 8 Sequential 134,037.2 ns 2,368.75 ns 2,432.53 ns 134,170.8 ns - - - 304 B
Read 1000 64 Sequential 135,275.9 ns 2,265.09 ns 2,118.77 ns 135,163.6 ns - - - 304 B
Read 1000 256 Sequential 137,855.6 ns 2,528.06 ns 2,364.75 ns 138,262.2 ns - - - 304 B

Write


BenchmarkDotNet=v0.12.1, OS=Windows 10.0.18362.1016 (1903/May2019Update/19H1)
Intel Core i9-9880H CPU 2.30GHz, 1 CPU, 8 logical and 8 physical cores
.NET Core SDK=5.0.301
  [Host]     : .NET Core 5.0.7 (CoreCLR 5.0.721.25508, CoreFX 5.0.721.25508), X64 RyuJIT
  DefaultJob : .NET Core 5.0.7 (CoreCLR 5.0.721.25508, CoreFX 5.0.721.25508), X64 RyuJIT
Method OpsPerTransaction ValueSize KeyOrder Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated
Write 1 8 Sequential 549.8 μs 10.84 μs 11.60 μs - - - 304 B
Write 1 64 Sequential 552.7 μs 10.96 μs 9.15 μs - - - 304 B
Write 1 256 Sequential 550.7 μs 8.26 μs 6.89 μs - - - 304 B
Write 100 8 Sequential 564.8 μs 10.87 μs 9.08 μs - - - 304 B
Write 100 64 Sequential 591.9 μs 10.99 μs 11.28 μs - - - 304 B
Write 100 256 Sequential 623.5 μs 12.34 μs 24.35 μs - - - 304 B
Write 1000 8 Sequential 808.3 μs 15.59 μs 16.69 μs - - - 304 B
Write 1000 64 Sequential 937.0 μs 18.42 μs 29.74 μs - - - 304 B
Write 1000 256 Sequential 1,157.4 μs 20.63 μs 17.23 μs - - - 304 B

Default Behavior

Read


BenchmarkDotNet=v0.12.1, OS=Windows 10.0.18362.1016 (1903/May2019Update/19H1)
Intel Core i9-9880H CPU 2.30GHz, 1 CPU, 8 logical and 8 physical cores
.NET Core SDK=5.0.301
  [Host]     : .NET Core 5.0.7 (CoreCLR 5.0.721.25508, CoreFX 5.0.721.25508), X64 RyuJIT
  DefaultJob : .NET Core 5.0.7 (CoreCLR 5.0.721.25508, CoreFX 5.0.721.25508), X64 RyuJIT
Method OpsPerTransaction ValueSize KeyOrder Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated
Read 1 8 Sequential 373.5 ns 7.47 ns 12.06 ns 0.0362 - - 304 B
Read 1 64 Sequential 385.6 ns 7.66 ns 9.96 ns 0.0362 - - 304 B
Read 1 256 Sequential 379.2 ns 7.27 ns 9.45 ns 0.0362 - - 304 B
Read 100 8 Sequential 9,910.4 ns 191.44 ns 268.37 ns 0.0305 - - 304 B
Read 100 64 Sequential 11,027.3 ns 149.12 ns 139.48 ns 0.0305 - - 304 B
Read 100 256 Sequential 11,399.1 ns 226.51 ns 222.47 ns 0.0305 - - 304 B
Read 1000 8 Sequential 133,331.1 ns 1,994.95 ns 1,959.31 ns - - - 304 B
Read 1000 64 Sequential 135,912.5 ns 2,343.05 ns 2,191.69 ns - - - 304 B
Read 1000 256 Sequential 136,153.0 ns 2,412.68 ns 2,256.82 ns - - - 304 B

Write


BenchmarkDotNet=v0.12.1, OS=Windows 10.0.18362.1016 (1903/May2019Update/19H1)
Intel Core i9-9880H CPU 2.30GHz, 1 CPU, 8 logical and 8 physical cores
.NET Core SDK=5.0.301
  [Host]     : .NET Core 5.0.7 (CoreCLR 5.0.721.25508, CoreFX 5.0.721.25508), X64 RyuJIT
  DefaultJob : .NET Core 5.0.7 (CoreCLR 5.0.721.25508, CoreFX 5.0.721.25508), X64 RyuJIT
Method OpsPerTransaction ValueSize KeyOrder Mean Error StdDev Median Gen 0 Gen 1 Gen 2 Allocated
Write 1 8 Sequential 569.6 μs 11.35 μs 27.20 μs 559.4 μs - - - 304 B
Write 1 64 Sequential 557.4 μs 7.02 μs 10.28 μs 556.2 μs - - - 304 B
Write 1 256 Sequential 551.1 μs 10.89 μs 9.66 μs 548.9 μs - - - 304 B
Write 100 8 Sequential 567.1 μs 6.14 μs 5.13 μs 566.9 μs - - - 304 B
Write 100 64 Sequential 595.6 μs 11.59 μs 13.35 μs 597.8 μs - - - 304 B
Write 100 256 Sequential 621.5 μs 11.01 μs 18.69 μs 619.5 μs - - - 304 B
Write 1000 8 Sequential 800.4 μs 11.07 μs 9.81 μs 800.7 μs - - - 304 B
Write 1000 64 Sequential 897.6 μs 8.55 μs 6.67 μs 899.6 μs - - - 304 B
Write 1000 256 Sequential 1,138.3 μs 17.40 μs 14.53 μs 1,139.6 μs - - - 304 B
AlgorithmsAreCool commented 3 years ago

That is great to hear, I remember reading that the autoresize was supposedly 10-15% slower. So this is all win