Closed arj03 closed 1 year ago
Part | Duration |
---|---|
Create 5000 new messages | 368.50ms |
Validate 5000 messages | 448.87ms |
Native to db format 5000 messages | 133.31ms |
Db to native format 5000 messages | 87.13ms |
Add 1000 elements | 522.64ms |
Add 1000 box1 msgs | 937.23ms |
Unbox 1000 box1 msgs first run | 220.98ms |
Unbox 1000 box1 msgs second run | 103.01ms |
Add 1000 box1 msgs | 904.02ms |
Query 1000 msgs first run | 39.47ms |
Query 1000 msgs second run | 22.98ms |
Add 1000 box2 msgs | 1297.41ms |
Unbox 1000 box2 msgs first run | 339.62ms |
Unbox 1000 box2 msgs second run | 151.35ms |
Migrate (+db1) | 11840.34ms |
Migrate (alone) | 3979.39ms |
Migrate (+db1 +db2) | 8259.00ms |
Migrate (+db2) | 6196.09ms |
Migrate continuation (+db2) | 1204.59ms |
Memory usage without indexes | 757.43 MB = 37.74 MB + etc |
Initial indexing | 1735.55ms |
Initial indexing maxcpu=86 | 5028.50ms |
Initial indexing compat | 677.80ms |
Two indexes updating concurrently | 2947.23ms |
Key one initial | 53.16ms |
Key two | 0.53ms |
Key one again | 1.96ms |
Reboot and key one again | 43.37ms |
Latest root posts | 1741.00ms |
Latest posts | 10.23ms |
Votes one initial | 1516.96ms |
Votes again | 0.59ms |
HasRoot | 1305.77ms |
HasRoot again | 0.31ms |
Author one posts | 1427.55ms |
Author two posts | 15.99ms |
Dedicated author one posts | 1449.55ms |
Dedicated author one posts again | 0.51ms |
DeleteFeed | 141854.89ms |
Maximum memory usage | 975.20 MB = 55.57 MB + etc |
Indexes folder size | 10.01mb |
Part | Duration |
---|---|
Create 5000 new messages | 365.83ms |
Validate 5000 messages | 447.78ms |
Native to db format 5000 messages | 114.84ms |
Db to native format 5000 messages | 82.26ms |
Add 1000 elements | 365.38ms |
Add 1000 box1 msgs | 930.05ms |
Unbox 1000 box1 msgs first run | 209.70ms |
Unbox 1000 box1 msgs second run | 106.42ms |
Add 1000 box1 msgs | 894.70ms |
Query 1000 msgs first run | 47.69ms |
Query 1000 msgs second run | 12.65ms |
Add 1000 box2 msgs | 1284.79ms |
Unbox 1000 box2 msgs first run | 316.14ms |
Unbox 1000 box2 msgs second run | 143.91ms |
Migrate (+db1) | 11830.30ms |
Migrate (alone) | 4061.75ms |
Migrate (+db1 +db2) | 7995.15ms |
Migrate (+db2) | 6448.79ms |
Migrate continuation (+db2) | 977.58ms |
Memory usage without indexes | 742.50 MB = 40.44 MB + etc |
Initial indexing | 1619.88ms |
Initial indexing maxcpu=86 | 5357.09ms |
Initial indexing compat | 747.66ms |
Two indexes updating concurrently | 2735.11ms |
Key one initial | 54.74ms |
Key two | 0.55ms |
Key one again | 1.48ms |
Reboot and key one again | 45.59ms |
Latest root posts | 1845.36ms |
Latest posts | 8.52ms |
Votes one initial | 1501.42ms |
Votes again | 0.62ms |
HasRoot | 1315.32ms |
HasRoot again | 0.33ms |
Author one posts | 1446.41ms |
Author two posts | 19.34ms |
Dedicated author one posts | 1456.70ms |
Dedicated author one posts again | 0.51ms |
DeleteFeed | 139037.52ms |
Maximum memory usage | 928.03 MB = 63.38 MB + etc |
Indexes folder size | 10.01mb |
I can see that DeleteFeed seems to be much slower than before. I'll try giving it a look, but it is a bit strange.
Part | Duration |
---|---|
Create 5000 new messages | 563.80ms |
Validate 5000 messages | 590.06ms |
Native to db format 5000 messages | 162.91ms |
Db to native format 5000 messages | 116.84ms |
Add 1000 elements | 521.55ms |
Add 1000 box1 msgs | 1372.09ms |
Unbox 1000 box1 msgs first run | 306.52ms |
Unbox 1000 box1 msgs second run | 136.78ms |
Add 1000 box1 msgs | 1355.47ms |
Query 1000 msgs first run | 53.03ms |
Query 1000 msgs second run | 26.39ms |
Add 1000 box2 msgs | 2029.51ms |
Unbox 1000 box2 msgs first run | 500.66ms |
Unbox 1000 box2 msgs second run | 243.05ms |
Migrate (+db1) | 16675.95ms |
Migrate (alone) | 5707.53ms |
Migrate (+db1 +db2) | 12727.93ms |
Migrate (+db2) | 8444.87ms |
Migrate continuation (+db2) | 1402.06ms |
Memory usage without indexes | 759.13 MB = 38.49 MB + etc |
Initial indexing | 2145.05ms |
Initial indexing maxcpu=86 | 6365.95ms |
Initial indexing compat | 1176.11ms |
Two indexes updating concurrently | 3876.36ms |
Key one initial | 61.33ms |
Key two | 1.37ms |
Key one again | 1.44ms |
Reboot and key one again | 64.79ms |
Latest root posts | 2419.49ms |
Latest posts | 16.97ms |
Votes one initial | 2014.44ms |
Votes again | 0.75ms |
HasRoot | 1735.90ms |
HasRoot again | 1.11ms |
Author one posts | 1863.42ms |
Author two posts | 28.08ms |
Dedicated author one posts | 1901.61ms |
Dedicated author one posts again | 0.67ms |
DeleteFeed | 147365.05ms |
Maximum memory usage | 930.62 MB = 61.07 MB + etc |
Indexes folder size | 10.01mb |
(Approved just in case it wasn't clear from discussion, but I know you might want to still work on this)
I pushed up a batch(1000), then the performance is the same as before. We need the same here. I'll do that as seperate PR.
Part | Duration |
---|---|
Create 5000 new messages | 372.92ms |
Validate 5000 messages | 483.66ms |
Native to db format 5000 messages | 108.28ms |
Db to native format 5000 messages | 83.85ms |
Add 1000 elements | 372.08ms |
Add 1000 box1 msgs | 1004.54ms |
Unbox 1000 box1 msgs first run | 217.13ms |
Unbox 1000 box1 msgs second run | 104.22ms |
Add 1000 box1 msgs | 975.55ms |
Query 1000 msgs first run | 40.71ms |
Query 1000 msgs second run | 13.26ms |
Add 1000 box2 msgs | 1309.02ms |
Unbox 1000 box2 msgs first run | 305.79ms |
Unbox 1000 box2 msgs second run | 139.82ms |
Migrate (+db1) | 11382.09ms |
Migrate (alone) | 4054.34ms |
Migrate (+db1 +db2) | 8455.81ms |
Migrate (+db2) | 5889.11ms |
Migrate continuation (+db2) | 885.89ms |
Memory usage without indexes | 753.36 MB = 40.53 MB + etc |
Initial indexing | 1591.65ms |
Initial indexing maxcpu=86 | 4839.90ms |
Initial indexing compat | 705.78ms |
Two indexes updating concurrently | 2839.31ms |
Key one initial | 52.21ms |
Key two | 0.50ms |
Key one again | 1.12ms |
Reboot and key one again | 42.67ms |
Latest root posts | 1719.27ms |
Latest posts | 9.31ms |
Votes one initial | 1459.52ms |
Votes again | 0.59ms |
HasRoot | 1316.85ms |
HasRoot again | 0.56ms |
Author one posts | 1407.09ms |
Author two posts | 21.13ms |
Dedicated author one posts | 1429.79ms |
Dedicated author one posts again | 0.58ms |
DeleteFeed | 5184.24ms |
Maximum memory usage | 996.18 MB = 59.98 MB + etc |
Indexes folder size | 10.01mb |
Part | Duration |
---|---|
Create 5000 new messages | 361.74ms |
Validate 5000 messages | 481.48ms |
Native to db format 5000 messages | 108.95ms |
Db to native format 5000 messages | 81.77ms |
Add 1000 elements | 520.85ms |
Add 1000 box1 msgs | 997.72ms |
Unbox 1000 box1 msgs first run | 224.21ms |
Unbox 1000 box1 msgs second run | 107.33ms |
Add 1000 box1 msgs | 970.64ms |
Query 1000 msgs first run | 35.68ms |
Query 1000 msgs second run | 19.94ms |
Add 1000 box2 msgs | 1290.31ms |
Unbox 1000 box2 msgs first run | 314.83ms |
Unbox 1000 box2 msgs second run | 145.18ms |
Migrate (+db1) | 10889.34ms |
Migrate (alone) | 3636.71ms |
Migrate (+db1 +db2) | 8113.63ms |
Migrate (+db2) | 5693.87ms |
Migrate continuation (+db2) | 841.02ms |
Memory usage without indexes | 745.05 MB = 41.42 MB + etc |
Initial indexing | 1559.67ms |
Initial indexing maxcpu=86 | 4950.33ms |
Initial indexing compat | 715.86ms |
Two indexes updating concurrently | 2770.41ms |
Key one initial | 45.96ms |
Key two | 1.36ms |
Key one again | 0.78ms |
Reboot and key one again | 49.91ms |
Latest root posts | 1673.38ms |
Latest posts | 9.06ms |
Votes one initial | 1459.72ms |
Votes again | 0.49ms |
HasRoot | 1276.52ms |
HasRoot again | 0.28ms |
Author one posts | 1389.99ms |
Author two posts | 14.63ms |
Dedicated author one posts | 1442.78ms |
Dedicated author one posts again | 0.51ms |
DeleteFeed | 4954.22ms |
Maximum memory usage | 1004.93 MB = 61.59 MB + etc |
Indexes folder size | 10.01mb |
This fixes #402. No need to load up all the offsets into memory if we just delete all of them anyway.