ssbc / ssb-db2

A new database for secure-scuttlebutt
47 stars 8 forks source link

dont allow queries to start when compact() just started #367

Closed staltz closed 2 years ago

staltz commented 2 years ago

Problem

compacting is set to true only when compactionProgress reports something, but that could take a while (it needs to write a block in AAOL), and in the meantime, someone could call ssb.db.query, which can proceed if compacting === false.

Solution

As soon as we call compact(), set compacting to true.

1st :x: 2nd :heavy_check_mark:

github-actions[bot] commented 2 years ago

Benchmark results

Part Duration
add 1000 elements 487ms
add 1000 private box1 elements 1162ms
unbox 1000 private box1 elements first run 120ms
unbox 1000 private box1 elements second run 104ms
add 1000 private box1 elements 1143ms
query 1000 elements first run 37ms
query 1000 elements second run 20ms
add 1000 private box2 elements 625ms
unbox 1000 private box2 elements first run 358ms
unbox 1000 private box2 elements second run 356ms
Migrate (+db1) 12093ms
Migrate (alone) 3924ms
Migrate (+db1 +db2) 8174ms
Migrate (+db2) 6449ms
Migrate continuation (+db2) 1016ms
Memory usage without indexes 724.62 MB = 32.10 MB + etc
Initial indexing 713ms
Initial indexing maxCpu=86 4231ms
Initial indexing compat 688ms
Two indexes updating concurrently 1136ms
key one initial 48ms
key two 0ms
key one again 1ms
reboot and key one again 43ms
latest root posts 824ms
latest posts 4ms
votes one initial 549ms
votes again 1ms
hasRoot 392ms
hasRoot again 1ms
author one posts 493ms
author two posts 18ms
dedicated author one posts 486ms
dedicated author one posts again 1ms
Maximum memory usage 1011.91 MB = 61.82 MB + etc
Indexes folder size 9.97mb
github-actions[bot] commented 2 years ago

Benchmark results

Part Duration
add 1000 elements 503ms
add 1000 private box1 elements 1383ms
unbox 1000 private box1 elements first run 168ms
unbox 1000 private box1 elements second run 110ms
add 1000 private box1 elements 1387ms
query 1000 elements first run 46ms
query 1000 elements second run 15ms
add 1000 private box2 elements 730ms
unbox 1000 private box2 elements first run 412ms
unbox 1000 private box2 elements second run 402ms
Migrate (+db1) 13406ms
Migrate (alone) 4856ms
Migrate (+db1 +db2) 10076ms
Migrate (+db2) 7501ms
Migrate continuation (+db2) 1077ms
Memory usage without indexes 748.34 MB = 30.84 MB + etc
Initial indexing 862ms
Initial indexing maxCpu=86 4831ms
Initial indexing compat 839ms
Two indexes updating concurrently 1172ms
key one initial 59ms
key two 3ms
key one again 1ms
reboot and key one again 56ms
latest root posts 891ms
latest posts 11ms
votes one initial 634ms
votes again 1ms
hasRoot 450ms
hasRoot again 1ms
author one posts 539ms
author two posts 22ms
dedicated author one posts 592ms
dedicated author one posts again 1ms
Maximum memory usage 1011.45 MB = 57.44 MB + etc
Indexes folder size 9.97mb
github-actions[bot] commented 2 years ago

Benchmark results

Part Duration
add 1000 elements 551ms
add 1000 private box1 elements 1080ms
unbox 1000 private box1 elements first run 123ms
unbox 1000 private box1 elements second run 76ms
add 1000 private box1 elements 950ms
query 1000 elements first run 35ms
query 1000 elements second run 10ms
add 1000 private box2 elements 551ms
unbox 1000 private box2 elements first run 311ms
unbox 1000 private box2 elements second run 293ms
Migrate (+db1) 10798ms
Migrate (alone) 3631ms
Migrate (+db1 +db2) 7092ms
Migrate (+db2) 5806ms
Migrate continuation (+db2) 929ms
Memory usage without indexes 740.77 MB = 31.08 MB + etc
Initial indexing 633ms
Initial indexing maxCpu=86 2754ms
Initial indexing compat 533ms
Two indexes updating concurrently 1015ms
key one initial 44ms
key two 1ms
key one again 0ms
reboot and key one again 46ms
latest root posts 732ms
latest posts 11ms
votes one initial 519ms
votes again 0ms
hasRoot 375ms
hasRoot again 0ms
author one posts 412ms
author two posts 14ms
dedicated author one posts 414ms
dedicated author one posts again 0ms
Maximum memory usage 1013.04 MB = 59.20 MB + etc
Indexes folder size 9.97mb