search

ssbc / ssb-db2

A new database for secure-scuttlebutt
47 stars 8 forks source link

deleteFeed to support removing OOO #417

Closed staltz closed 1 year ago

staltz commented 1 year ago

Context

https://github.com/ssbc/ssb-replication-scheduler/issues/14

Problem

deleteFeed doesn't support deleting messages that were added with OOO, because we have a state check, and addOOO has been recently updated #412 to not update the Base state.

Solution

Drop the state check in deleteFeed.

Add tests.

github-actions[bot] commented 1 year ago

Benchmark results

Part Duration
Create 5000 new messages 365.25ms
Validate 5000 messages 480.18ms
Native to db format 5000 messages 108.89ms
Db to native format 5000 messages 85.59ms
Add 1000 elements 494.55ms
Add 1000 box1 msgs 1000.61ms
Unbox 1000 box1 msgs first run 215.75ms
Unbox 1000 box1 msgs second run 106.79ms
Add 1000 box1 msgs 961.55ms
Query 1000 msgs first run 51.68ms
Query 1000 msgs second run 12.16ms
Add 1000 box2 msgs 1271.58ms
Unbox 1000 box2 msgs first run 315.94ms
Unbox 1000 box2 msgs second run 146.10ms
Migrate (+db1) 10673.75ms
Migrate (alone) 3564.19ms
Migrate (+db1 +db2) 8202.69ms
Migrate (+db2) 5657.83ms
Migrate continuation (+db2) 832.68ms
Memory usage without indexes 755.13 MB = 38.65 MB + etc
Initial indexing 1544.82ms
Initial indexing maxcpu=86 3927.84ms
Initial indexing compat 733.11ms
Two indexes updating concurrently 2709.03ms
Key one initial 48.69ms
Key two 1.81ms
Key one again 0.99ms
Reboot and key one again 47.59ms
Latest root posts 1700.88ms
Latest posts 8.10ms
Votes one initial 1436.19ms
Votes again 0.54ms
HasRoot 1300.33ms
HasRoot again 0.34ms
Author one posts 1401.04ms
Author two posts 17.68ms
Dedicated author one posts 1399.16ms
Dedicated author one posts again 0.56ms
DeleteFeed 4986.93ms
Maximum memory usage 993.84 MB = 60.36 MB + etc
Indexes folder size 10.01mb
github-actions[bot] commented 1 year ago

Benchmark results

Part Duration
Create 5000 new messages 448.41ms
Validate 5000 messages 521.82ms
Native to db format 5000 messages 145.41ms
Db to native format 5000 messages 97.99ms
Add 1000 elements 380.60ms
Add 1000 box1 msgs 1106.29ms
Unbox 1000 box1 msgs first run 246.35ms
Unbox 1000 box1 msgs second run 124.33ms
Add 1000 box1 msgs 1056.72ms
Query 1000 msgs first run 61.98ms
Query 1000 msgs second run 18.04ms
Add 1000 box2 msgs 1483.95ms
Unbox 1000 box2 msgs first run 387.10ms
Unbox 1000 box2 msgs second run 194.26ms
Migrate (+db1) 13290.18ms
Migrate (alone) 4698.11ms
Migrate (+db1 +db2) 9882.37ms
Migrate (+db2) 7338.61ms
Migrate continuation (+db2) 1265.23ms
Memory usage without indexes 741.71 MB = 36.89 MB + etc
Initial indexing 1878.65ms
Initial indexing maxcpu=86 6404.96ms
Initial indexing compat 982.85ms
Two indexes updating concurrently 3282.58ms
Key one initial 59.67ms
Key two 1.33ms
Key one again 0.93ms
Reboot and key one again 54.08ms
Latest root posts 2004.96ms
Latest posts 10.84ms
Votes one initial 1647.66ms
Votes again 0.62ms
HasRoot 1468.29ms
HasRoot again 0.34ms
Author one posts 1566.30ms
Author two posts 21.09ms
Dedicated author one posts 1601.89ms
Dedicated author one posts again 0.57ms
DeleteFeed 6768.55ms
Maximum memory usage 1009.55 MB = 58.21 MB + etc
Indexes folder size 10.01mb