ssbc / ssb-db2

A new database for secure-scuttlebutt
47 stars 8 forks source link

Recommend toPullStream over toCallback #408

Closed staltz closed 1 year ago

staltz commented 1 year ago

Just a quick tip for other developers in the future, that toCallback can be an OOM footgun.

github-actions[bot] commented 1 year ago

Benchmark results

Part Duration
Create 5000 new messages 370.81ms
Validate 5000 messages 482.14ms
Native to db format 5000 messages 123.01ms
Db to native format 5000 messages 82.14ms
Add 1000 elements 495.24ms
Add 1000 box1 msgs 1007.51ms
Unbox 1000 box1 msgs first run 219.74ms
Unbox 1000 box1 msgs second run 111.08ms
Add 1000 box1 msgs 968.12ms
Query 1000 msgs first run 46.95ms
Query 1000 msgs second run 12.36ms
Add 1000 box2 msgs 1290.97ms
Unbox 1000 box2 msgs first run 334.16ms
Unbox 1000 box2 msgs second run 168.91ms
Migrate (+db1) 10915.17ms
Migrate (alone) 3606.63ms
Migrate (+db1 +db2) 8270.73ms
Migrate (+db2) 5659.46ms
Migrate continuation (+db2) 916.72ms
Memory usage without indexes 760.54 MB = 40.10 MB + etc
Initial indexing 1563.19ms
Initial indexing maxcpu=86 4603.04ms
Initial indexing compat 731.28ms
Two indexes updating concurrently 2780.93ms
Key one initial 53.39ms
Key two 1.17ms
Key one again 1.93ms
Reboot and key one again 43.84ms
Latest root posts 1701.24ms
Latest posts 9.59ms
Votes one initial 1460.79ms
Votes again 0.49ms
HasRoot 1284.20ms
HasRoot again 0.29ms
Author one posts 1382.11ms
Author two posts 17.28ms
Dedicated author one posts 1407.10ms
Dedicated author one posts again 0.60ms
DeleteFeed 4976.61ms
Maximum memory usage 1007.68 MB = 60.41 MB + etc
Indexes folder size 10.01mb
github-actions[bot] commented 1 year ago

Benchmark results

Part Duration
Create 5000 new messages 377.69ms
Validate 5000 messages 476.50ms
Native to db format 5000 messages 111.60ms
Db to native format 5000 messages 82.17ms
Add 1000 elements 493.20ms
Add 1000 box1 msgs 998.44ms
Unbox 1000 box1 msgs first run 217.68ms
Unbox 1000 box1 msgs second run 107.97ms
Add 1000 box1 msgs 963.25ms
Query 1000 msgs first run 38.34ms
Query 1000 msgs second run 21.71ms
Add 1000 box2 msgs 1273.83ms
Unbox 1000 box2 msgs first run 324.70ms
Unbox 1000 box2 msgs second run 163.39ms
Migrate (+db1) 10687.68ms
Migrate (alone) 3641.13ms
Migrate (+db1 +db2) 8225.73ms
Migrate (+db2) 5583.86ms
Migrate continuation (+db2) 901.73ms
Memory usage without indexes 748.75 MB = 36.70 MB + etc
Initial indexing 1524.09ms
Initial indexing maxcpu=86 4450.24ms
Initial indexing compat 631.87ms
Two indexes updating concurrently 2782.25ms
Key one initial 51.49ms
Key two 0.98ms
Key one again 1.79ms
Reboot and key one again 43.09ms
Latest root posts 1644.83ms
Latest posts 10.00ms
Votes one initial 1431.71ms
Votes again 0.46ms
HasRoot 1265.54ms
HasRoot again 0.30ms
Author one posts 1380.58ms
Author two posts 15.29ms
Dedicated author one posts 1394.23ms
Dedicated author one posts again 0.43ms
DeleteFeed 4833.24ms
Maximum memory usage 1011.74 MB = 63.73 MB + etc
Indexes folder size 10.01mb