Closed staltz closed 2 years ago
Part | Speed | Heap Change | Samples |
---|---|---|---|
Count 1 big index (3rd run) | 0.38ms ± 0.03ms | -1.8 kB ± 10.57 kB | 59 |
Create an index twice concurrently | 500.15ms ± 2.67ms | 18.21 kB ± 25.39 kB | 109 |
Load core indexes | 0.8ms ± 0.01ms | 92.7 B ± 181.62 B | 9685 |
Load two indexes concurrently | 407.67ms ± 2.56ms | -83.75 kB ± 106.33 kB | 24 |
Paginate 10 results | 26.35ms ± 1.64ms | 11.51 kB ± 16.76 kB | 29 |
Paginate 20000 msgs with pageSize=5 | 6103.73ms ± 86.69ms | 145.33 kB ± 973.87 kB | 5 |
Paginate 20000 msgs with pageSize=500 | 480.78ms ± 2.46ms | 231.13 kB ± 415.86 kB | 23 |
Query 1 big index (1st run) | 742.65ms ± 11.93ms | -2.83 kB ± 38.49 kB | 74 |
Query 1 big index (2nd run) | 251.61ms ± 0.62ms | 1.71 kB ± 10.18 kB | 59 |
Query 3 indexes (1st run) | 1068.12ms ± 14.93ms | -52.37 kB ± 150.24 kB | 50 |
Query 3 indexes (2nd run) | 218.64ms ± 1.08ms | 21.41 kB ± 24.41 kB | 62 |
Query a prefix map (1st run) | 225.85ms ± 1.94ms | 28.97 kB ± 534.77 kB | 26 |
Query a prefix map (2nd run) | 14ms ± 0.7ms | -12.99 kB ± 45.61 kB | 26 |
Context: https://github.com/ssb-ngi-pointer/jitdb/issues/199
Problem 1: we need to know when is indexing ongoing and/or when is querying ongoing, because we don't want compact to happen in parallel with those.
Problem 2: if compaction is ongoing, we can't allow JITDB queries to start.
Problem 3:
reindex
has to rebuild also the core indexes.Solution:
indexingActive
andqueriesActive
which are Obz's that emit numbers of how many are active at the moment.reindex
rebuilds also the core indexesThese changes are already used and tested in https://github.com/ssb-ngi-pointer/ssb-db2/pull/339