search

clockworklabs / SpacetimeDB

Multiplayer at the speed of light
https://spacetimedb.com
Other
4.17k stars 102 forks source link

Use a copy-on-write structure for `RelValue` #869

Closed Centril closed 3 months ago

Centril commented 3 months ago

Description of Changes

The new definition is:

pub enum RelValue<'a> {
    Row(RowRef<'a>),
    Projection(ProductValue),
}

The PR also removes other unnecessary clones, including some notable ones, e.g., MemTable::next.

API and ABI breaking changes

None

Expected complexity level and risk

2

gefjon commented 3 months ago

benchmarks please

github-actions[bot] commented 3 months ago
Benchmark results # Benchmark Report Legend: - `load`: number of rows pre-loaded into the database - `count`: number of rows touched by the transaction - index types: - `unique`: a single index on the `id` column - `non_unique`: no indexes - `multi_index`: non-unique index on every column - schemas: - `person(id: u32, name: String, age: u64)` - `location(id: u32, x: u64, y: u64)` All throughputs are single-threaded. ## Empty transaction | db | on disk | new latency | old latency | new throughput | old throughput | |-------------|---------|---------------|---------------|----------------|----------------| | sqlite | 💿 | - | 414.3±1.60ns | - | - | | sqlite | 🧠 | - | 407.1±0.89ns | - | - | | stdb_module | 💿 | 18.9±1.35µs | 19.0±1.61µs | - | - | | stdb_module | 🧠 | 18.2±1.08µs | 18.2±0.87µs | - | - | | stdb_raw | 💿 | 767.9±1.28ns | 771.7±1.31ns | - | - | | stdb_raw | 🧠 | 715.1±0.79ns | 723.3±2.09ns | - | - | ## Single-row insertions | db | on disk | schema | index type | load | new latency | old latency | new throughput | old throughput | |-------------|---------|----------|-------------|------|------------------|-----------------|----------------|----------------| | sqlite | 💿 | location | multi_index | 0 | - | 14.6±0.16µs | - | 66.8 Ktx/sec | | sqlite | 💿 | location | multi_index | 1000 | - | 15.8±0.04µs | - | 62.0 Ktx/sec | | sqlite | 💿 | location | non_unique | 0 | - | 7.0±0.08µs | - | 139.0 Ktx/sec | | sqlite | 💿 | location | non_unique | 1000 | - | 7.7±2.35µs | - | 126.4 Ktx/sec | | sqlite | 💿 | location | unique | 0 | - | 7.0±0.06µs | - | 139.3 Ktx/sec | | sqlite | 💿 | location | unique | 1000 | - | 7.0±0.01µs | - | 140.0 Ktx/sec | | sqlite | 💿 | person | multi_index | 0 | - | 14.1±0.05µs | - | 69.0 Ktx/sec | | sqlite | 💿 | person | multi_index | 1000 | - | 15.9±0.18µs | - | 61.3 Ktx/sec | | sqlite | 💿 | person | non_unique | 0 | - | 7.1±0.02µs | - | 136.7 Ktx/sec | | sqlite | 💿 | person | non_unique | 1000 | - | 7.2±0.04µs | - | 136.5 Ktx/sec | | sqlite | 💿 | person | unique | 0 | - | 7.2±0.39µs | - | 136.4 Ktx/sec | | sqlite | 💿 | person | unique | 1000 | - | 7.2±0.06µs | - | 136.2 Ktx/sec | | sqlite | 🧠 | location | multi_index | 0 | - | 4.1±0.01µs | - | 240.1 Ktx/sec | | sqlite | 🧠 | location | multi_index | 1000 | - | 5.2±0.01µs | - | 187.9 Ktx/sec | | sqlite | 🧠 | location | non_unique | 0 | - | 1811.8±5.91ns | - | 539.0 Ktx/sec | | sqlite | 🧠 | location | non_unique | 1000 | - | 1853.4±4.49ns | - | 526.9 Ktx/sec | | sqlite | 🧠 | location | unique | 0 | - | 1804.7±4.55ns | - | 541.1 Ktx/sec | | sqlite | 🧠 | location | unique | 1000 | - | 1896.4±6.61ns | - | 515.0 Ktx/sec | | sqlite | 🧠 | person | multi_index | 0 | - | 3.6±0.01µs | - | 270.0 Ktx/sec | | sqlite | 🧠 | person | multi_index | 1000 | - | 5.4±0.04µs | - | 180.6 Ktx/sec | | sqlite | 🧠 | person | non_unique | 0 | - | 1882.9±5.65ns | - | 518.6 Ktx/sec | | sqlite | 🧠 | person | non_unique | 1000 | - | 1951.1±9.85ns | - | 500.5 Ktx/sec | | sqlite | 🧠 | person | unique | 0 | - | 1867.9±3.20ns | - | 522.8 Ktx/sec | | sqlite | 🧠 | person | unique | 1000 | - | 1986.5±12.79ns | - | 491.6 Ktx/sec | | stdb_module | 💿 | location | multi_index | 0 | 47.3±3.43µs | 46.5±3.91µs | 20.7 Ktx/sec | 21.0 Ktx/sec | | stdb_module | 💿 | location | multi_index | 1000 | 85.4±1.98µs | 82.3±3.72µs | 11.4 Ktx/sec | 11.9 Ktx/sec | | stdb_module | 💿 | location | non_unique | 0 | 43.8±2.99µs | 43.0±3.35µs | 22.3 Ktx/sec | 22.7 Ktx/sec | | stdb_module | 💿 | location | non_unique | 1000 | 76.5±3.45µs | 55.0±0.42µs | 12.8 Ktx/sec | 17.8 Ktx/sec | | stdb_module | 💿 | location | unique | 0 | 44.4±2.27µs | 44.4±2.88µs | 22.0 Ktx/sec | 22.0 Ktx/sec | | stdb_module | 💿 | location | unique | 1000 | 73.2±2.87µs | 82.9±0.40µs | 13.3 Ktx/sec | 11.8 Ktx/sec | | stdb_module | 💿 | person | multi_index | 0 | 51.5±13.52µs | 49.5±4.95µs | 19.0 Ktx/sec | 19.7 Ktx/sec | | stdb_module | 💿 | person | multi_index | 1000 | 93.4±6.94µs | 119.3±1.68µs | 10.5 Ktx/sec | 8.2 Ktx/sec | | stdb_module | 💿 | person | non_unique | 0 | 47.3±3.38µs | 48.1±4.47µs | 20.6 Ktx/sec | 20.3 Ktx/sec | | stdb_module | 💿 | person | non_unique | 1000 | 110.7±1.43µs | 112.1±1.85µs | 8.8 Ktx/sec | 8.7 Ktx/sec | | stdb_module | 💿 | person | unique | 0 | 52.2±4.51µs | 53.3±3.38µs | 18.7 Ktx/sec | 18.3 Ktx/sec | | stdb_module | 💿 | person | unique | 1000 | 159.6±840.88µs | 114.9±1.97µs | 6.1 Ktx/sec | 8.5 Ktx/sec | | stdb_module | 🧠 | location | multi_index | 0 | 34.7±1.26µs | 36.7±2.47µs | 28.2 Ktx/sec | 26.6 Ktx/sec | | stdb_module | 🧠 | location | multi_index | 1000 | 66.0±2.83µs | 43.5±4.26µs | 14.8 Ktx/sec | 22.4 Ktx/sec | | stdb_module | 🧠 | location | non_unique | 0 | 30.9±1.42µs | 32.2±2.21µs | 31.6 Ktx/sec | 30.3 Ktx/sec | | stdb_module | 🧠 | location | non_unique | 1000 | 44.3±3.11µs | 36.8±2.78µs | 22.1 Ktx/sec | 26.5 Ktx/sec | | stdb_module | 🧠 | location | unique | 0 | 32.3±0.92µs | 35.1±2.21µs | 30.2 Ktx/sec | 27.8 Ktx/sec | | stdb_module | 🧠 | location | unique | 1000 | 40.7±4.30µs | 45.9±5.15µs | 24.0 Ktx/sec | 21.3 Ktx/sec | | stdb_module | 🧠 | person | multi_index | 0 | 36.1±2.79µs | 39.3±2.78µs | 27.0 Ktx/sec | 24.9 Ktx/sec | | stdb_module | 🧠 | person | multi_index | 1000 | 110.0±2.15µs | 72.4±0.48µs | 8.9 Ktx/sec | 13.5 Ktx/sec | | stdb_module | 🧠 | person | non_unique | 0 | 37.6±2.90µs | 36.3±3.99µs | 26.0 Ktx/sec | 26.9 Ktx/sec | | stdb_module | 🧠 | person | non_unique | 1000 | 53.7±4.38µs | 95.5±1.29µs | 18.2 Ktx/sec | 10.2 Ktx/sec | | stdb_module | 🧠 | person | unique | 0 | 36.8±4.84µs | 37.5±3.39µs | 26.5 Ktx/sec | 26.0 Ktx/sec | | stdb_module | 🧠 | person | unique | 1000 | 97.3±7.77µs | 93.0±4.74µs | 10.0 Ktx/sec | 10.5 Ktx/sec | | stdb_raw | 💿 | location | multi_index | 0 | 5.8±0.01µs | 5.7±0.01µs | 167.5 Ktx/sec | 172.5 Ktx/sec | | stdb_raw | 💿 | location | multi_index | 1000 | 17.2±27.40µs | 19.2±34.57µs | 56.6 Ktx/sec | 51.0 Ktx/sec | | stdb_raw | 💿 | location | non_unique | 0 | 5.1±0.04µs | 5.1±0.01µs | 190.3 Ktx/sec | 192.6 Ktx/sec | | stdb_raw | 💿 | location | non_unique | 1000 | 14.8±24.89µs | 12.4±17.86µs | 66.1 Ktx/sec | 78.7 Ktx/sec | | stdb_raw | 💿 | location | unique | 0 | 5.6±0.02µs | 5.6±0.01µs | 173.3 Ktx/sec | 175.9 Ktx/sec | | stdb_raw | 💿 | location | unique | 1000 | 13.5±17.20µs | 15.6±24.51µs | 72.3 Ktx/sec | 62.7 Ktx/sec | | stdb_raw | 💿 | person | multi_index | 0 | 7.2±0.01µs | 7.1±0.04µs | 135.8 Ktx/sec | 137.5 Ktx/sec | | stdb_raw | 💿 | person | multi_index | 1000 | 31.3±208.09µs | 26.2±159.09µs | 31.2 Ktx/sec | 37.3 Ktx/sec | | stdb_raw | 💿 | person | non_unique | 0 | 6.4±0.01µs | 6.3±0.02µs | 153.7 Ktx/sec | 154.0 Ktx/sec | | stdb_raw | 💿 | person | non_unique | 1000 | 8.6±0.10µs | 8.4±0.08µs | 113.9 Ktx/sec | 116.5 Ktx/sec | | stdb_raw | 💿 | person | unique | 0 | 6.9±0.35µs | 6.8±0.04µs | 141.3 Ktx/sec | 143.6 Ktx/sec | | stdb_raw | 💿 | person | unique | 1000 | 9.5±0.11µs | 9.5±0.17µs | 102.5 Ktx/sec | 102.9 Ktx/sec | | stdb_raw | 🧠 | location | multi_index | 0 | 3.2±0.00µs | 3.2±0.02µs | 303.6 Ktx/sec | 304.2 Ktx/sec | | stdb_raw | 🧠 | location | multi_index | 1000 | 4.6±0.02µs | 4.7±0.02µs | 210.3 Ktx/sec | 208.8 Ktx/sec | | stdb_raw | 🧠 | location | non_unique | 0 | 2.7±0.01µs | 2.6±0.01µs | 366.3 Ktx/sec | 369.8 Ktx/sec | | stdb_raw | 🧠 | location | non_unique | 1000 | 3.4±0.01µs | 3.3±0.01µs | 286.9 Ktx/sec | 293.0 Ktx/sec | | stdb_raw | 🧠 | location | unique | 0 | 3.1±0.01µs | 3.1±0.01µs | 316.2 Ktx/sec | 317.6 Ktx/sec | | stdb_raw | 🧠 | location | unique | 1000 | 4.4±0.01µs | 4.3±0.01µs | 224.2 Ktx/sec | 226.7 Ktx/sec | | stdb_raw | 🧠 | person | multi_index | 0 | 4.0±0.02µs | 4.0±0.02µs | 243.3 Ktx/sec | 244.1 Ktx/sec | | stdb_raw | 🧠 | person | multi_index | 1000 | 6.0±0.03µs | 5.9±0.03µs | 163.7 Ktx/sec | 166.4 Ktx/sec | | stdb_raw | 🧠 | person | non_unique | 0 | 3.3±0.01µs | 3.3±0.01µs | 294.3 Ktx/sec | 294.8 Ktx/sec | | stdb_raw | 🧠 | person | non_unique | 1000 | 4.2±0.03µs | 4.3±0.03µs | 230.0 Ktx/sec | 229.2 Ktx/sec | | stdb_raw | 🧠 | person | unique | 0 | 3.8±0.01µs | 3.8±0.01µs | 257.5 Ktx/sec | 257.1 Ktx/sec | | stdb_raw | 🧠 | person | unique | 1000 | 5.1±0.04µs | 5.2±0.06µs | 190.9 Ktx/sec | 189.1 Ktx/sec | ## Multi-row insertions | db | on disk | schema | index type | load | count | new latency | old latency | new throughput | old throughput | |-------------|---------|----------|-------------|------|-------|------------------|------------------|----------------|----------------| | sqlite | 💿 | location | multi_index | 0 | 100 | - | 132.1±1.59µs | - | 7.4 Ktx/sec | | sqlite | 💿 | location | multi_index | 1000 | 100 | - | 199.2±0.25µs | - | 4.9 Ktx/sec | | sqlite | 💿 | location | non_unique | 0 | 100 | - | 50.0±0.48µs | - | 19.5 Ktx/sec | | sqlite | 💿 | location | non_unique | 1000 | 100 | - | 52.4±0.12µs | - | 18.6 Ktx/sec | | sqlite | 💿 | location | unique | 0 | 100 | - | 51.8±1.15µs | - | 18.8 Ktx/sec | | sqlite | 💿 | location | unique | 1000 | 100 | - | 55.9±0.56µs | - | 17.5 Ktx/sec | | sqlite | 💿 | person | multi_index | 0 | 100 | - | 116.5±0.28µs | - | 8.4 Ktx/sec | | sqlite | 💿 | person | multi_index | 1000 | 100 | - | 231.9±0.34µs | - | 4.2 Ktx/sec | | sqlite | 💿 | person | non_unique | 0 | 100 | - | 48.2±0.49µs | - | 20.3 Ktx/sec | | sqlite | 💿 | person | non_unique | 1000 | 100 | - | 59.3±0.20µs | - | 16.5 Ktx/sec | | sqlite | 💿 | person | unique | 0 | 100 | - | 50.6±0.19µs | - | 19.3 Ktx/sec | | sqlite | 💿 | person | unique | 1000 | 100 | - | 54.9±0.17µs | - | 17.8 Ktx/sec | | sqlite | 🧠 | location | multi_index | 0 | 100 | - | 122.6±0.18µs | - | 8.0 Ktx/sec | | sqlite | 🧠 | location | multi_index | 1000 | 100 | - | 169.2±0.14µs | - | 5.8 Ktx/sec | | sqlite | 🧠 | location | non_unique | 0 | 100 | - | 44.2±0.31µs | - | 22.1 Ktx/sec | | sqlite | 🧠 | location | non_unique | 1000 | 100 | - | 44.7±0.12µs | - | 21.8 Ktx/sec | | sqlite | 🧠 | location | unique | 0 | 100 | - | 45.5±0.23µs | - | 21.4 Ktx/sec | | sqlite | 🧠 | location | unique | 1000 | 100 | - | 49.2±0.44µs | - | 19.9 Ktx/sec | | sqlite | 🧠 | person | multi_index | 0 | 100 | - | 107.6±0.23µs | - | 9.1 Ktx/sec | | sqlite | 🧠 | person | multi_index | 1000 | 100 | - | 189.9±0.16µs | - | 5.1 Ktx/sec | | sqlite | 🧠 | person | non_unique | 0 | 100 | - | 42.1±0.24µs | - | 23.2 Ktx/sec | | sqlite | 🧠 | person | non_unique | 1000 | 100 | - | 45.9±0.41µs | - | 21.3 Ktx/sec | | sqlite | 🧠 | person | unique | 0 | 100 | - | 44.3±0.34µs | - | 22.1 Ktx/sec | | sqlite | 🧠 | person | unique | 1000 | 100 | - | 47.3±0.51µs | - | 20.7 Ktx/sec | | stdb_module | 💿 | location | multi_index | 0 | 100 | 450.4±1.58µs | 439.5±14.75µs | 2.2 Ktx/sec | 2.2 Ktx/sec | | stdb_module | 💿 | location | multi_index | 1000 | 100 | 526.1±6.20µs | 465.2±84.76µs | 1900 tx/sec | 2.1 Ktx/sec | | stdb_module | 💿 | location | non_unique | 0 | 100 | 335.1±34.51µs | 415.0±66.24µs | 2.9 Ktx/sec | 2.4 Ktx/sec | | stdb_module | 💿 | location | non_unique | 1000 | 100 | 274.2±39.45µs | 250.7±0.74µs | 3.6 Ktx/sec | 3.9 Ktx/sec | | stdb_module | 💿 | location | unique | 0 | 100 | 465.1±7.51µs | 410.1±36.30µs | 2.1 Ktx/sec | 2.4 Ktx/sec | | stdb_module | 💿 | location | unique | 1000 | 100 | 468.4±47.31µs | 505.1±11.21µs | 2.1 Ktx/sec | 1979 tx/sec | | stdb_module | 💿 | person | multi_index | 0 | 100 | 784.0±104.93µs | 690.2±107.11µs | 1275 tx/sec | 1448 tx/sec | | stdb_module | 💿 | person | multi_index | 1000 | 100 | 888.1±1.25µs | 701.1±52.11µs | 1126 tx/sec | 1426 tx/sec | | stdb_module | 💿 | person | non_unique | 0 | 100 | 648.0±11.70µs | 681.1±10.80µs | 1543 tx/sec | 1468 tx/sec | | stdb_module | 💿 | person | non_unique | 1000 | 100 | 701.1±34.91µs | 553.1±86.14µs | 1426 tx/sec | 1808 tx/sec | | stdb_module | 💿 | person | unique | 0 | 100 | 683.6±0.53µs | 727.3±0.68µs | 1462 tx/sec | 1374 tx/sec | | stdb_module | 💿 | person | unique | 1000 | 100 | 805.5±2.49µs | 721.9±3.10µs | 1241 tx/sec | 1385 tx/sec | | stdb_module | 🧠 | location | multi_index | 0 | 100 | 272.7±45.00µs | 244.3±1.55µs | 3.6 Ktx/sec | 4.0 Ktx/sec | | stdb_module | 🧠 | location | multi_index | 1000 | 100 | 444.6±0.76µs | 389.3±29.32µs | 2.2 Ktx/sec | 2.5 Ktx/sec | | stdb_module | 🧠 | location | non_unique | 0 | 100 | 208.1±7.09µs | 235.7±37.39µs | 4.7 Ktx/sec | 4.1 Ktx/sec | | stdb_module | 🧠 | location | non_unique | 1000 | 100 | 213.8±0.39µs | 266.2±54.33µs | 4.6 Ktx/sec | 3.7 Ktx/sec | | stdb_module | 🧠 | location | unique | 0 | 100 | 325.1±74.30µs | 264.3±27.45µs | 3.0 Ktx/sec | 3.7 Ktx/sec | | stdb_module | 🧠 | location | unique | 1000 | 100 | 424.6±0.65µs | 257.0±0.24µs | 2.3 Ktx/sec | 3.8 Ktx/sec | | stdb_module | 🧠 | person | multi_index | 0 | 100 | 536.1±6.75µs | 556.4±0.27µs | 1865 tx/sec | 1797 tx/sec | | stdb_module | 🧠 | person | multi_index | 1000 | 100 | 360.4±0.34µs | 360.6±0.37µs | 2.7 Ktx/sec | 2.7 Ktx/sec | | stdb_module | 🧠 | person | non_unique | 0 | 100 | 430.1±15.62µs | 454.6±12.05µs | 2.3 Ktx/sec | 2.1 Ktx/sec | | stdb_module | 🧠 | person | non_unique | 1000 | 100 | 512.9±0.82µs | 437.4±0.63µs | 1949 tx/sec | 2.2 Ktx/sec | | stdb_module | 🧠 | person | unique | 0 | 100 | 462.0±35.90µs | 485.0±0.69µs | 2.1 Ktx/sec | 2.0 Ktx/sec | | stdb_module | 🧠 | person | unique | 1000 | 100 | 528.2±45.20µs | 422.8±98.75µs | 1893 tx/sec | 2.3 Ktx/sec | | stdb_raw | 💿 | location | multi_index | 0 | 100 | 162.2±1.03µs | 159.7±0.13µs | 6.0 Ktx/sec | 6.1 Ktx/sec | | stdb_raw | 💿 | location | multi_index | 1000 | 100 | 175.8±0.41µs | 175.1±0.31µs | 5.6 Ktx/sec | 5.6 Ktx/sec | | stdb_raw | 💿 | location | non_unique | 0 | 100 | 121.0±1.39µs | 120.3±0.03µs | 8.1 Ktx/sec | 8.1 Ktx/sec | | stdb_raw | 💿 | location | non_unique | 1000 | 100 | 143.9±67.85µs | 122.7±0.33µs | 6.8 Ktx/sec | 8.0 Ktx/sec | | stdb_raw | 💿 | location | unique | 0 | 100 | 156.5±0.54µs | 155.4±0.21µs | 6.2 Ktx/sec | 6.3 Ktx/sec | | stdb_raw | 💿 | location | unique | 1000 | 100 | 162.5±0.68µs | 173.1±36.31µs | 6.0 Ktx/sec | 5.6 Ktx/sec | | stdb_raw | 💿 | person | multi_index | 0 | 100 | 279.9±6.31µs | 283.5±0.13µs | 3.5 Ktx/sec | 3.4 Ktx/sec | | stdb_raw | 💿 | person | multi_index | 1000 | 100 | 310.4±67.94µs | 292.7±0.35µs | 3.1 Ktx/sec | 3.3 Ktx/sec | | stdb_raw | 💿 | person | non_unique | 0 | 100 | 210.8±0.11µs | 217.2±0.17µs | 4.6 Ktx/sec | 4.5 Ktx/sec | | stdb_raw | 💿 | person | non_unique | 1000 | 100 | 230.3±60.10µs | 218.1±0.24µs | 4.2 Ktx/sec | 4.5 Ktx/sec | | stdb_raw | 💿 | person | unique | 0 | 100 | 246.7±0.12µs | 253.8±0.29µs | 4.0 Ktx/sec | 3.8 Ktx/sec | | stdb_raw | 💿 | person | unique | 1000 | 100 | 256.0±0.48µs | 260.4±0.22µs | 3.8 Ktx/sec | 3.7 Ktx/sec | | stdb_raw | 🧠 | location | multi_index | 0 | 100 | 121.5±0.15µs | 120.0±0.26µs | 8.0 Ktx/sec | 8.1 Ktx/sec | | stdb_raw | 🧠 | location | multi_index | 1000 | 100 | 135.4±0.11µs | 134.0±0.11µs | 7.2 Ktx/sec | 7.3 Ktx/sec | | stdb_raw | 🧠 | location | non_unique | 0 | 100 | 81.3±0.05µs | 81.2±0.08µs | 12.0 Ktx/sec | 12.0 Ktx/sec | | stdb_raw | 🧠 | location | non_unique | 1000 | 100 | 82.4±0.06µs | 82.3±0.06µs | 11.9 Ktx/sec | 11.9 Ktx/sec | | stdb_raw | 🧠 | location | unique | 0 | 100 | 117.3±0.13µs | 116.1±0.28µs | 8.3 Ktx/sec | 8.4 Ktx/sec | | stdb_raw | 🧠 | location | unique | 1000 | 100 | 121.9±0.05µs | 122.4±0.38µs | 8.0 Ktx/sec | 8.0 Ktx/sec | | stdb_raw | 🧠 | person | multi_index | 0 | 100 | 188.1±0.16µs | 194.1±0.13µs | 5.2 Ktx/sec | 5.0 Ktx/sec | | stdb_raw | 🧠 | person | multi_index | 1000 | 100 | 199.5±0.52µs | 205.1±0.17µs | 4.9 Ktx/sec | 4.8 Ktx/sec | | stdb_raw | 🧠 | person | non_unique | 0 | 100 | 123.6±0.09µs | 130.5±0.39µs | 7.9 Ktx/sec | 7.5 Ktx/sec | | stdb_raw | 🧠 | person | non_unique | 1000 | 100 | 124.1±0.09µs | 130.7±0.32µs | 7.9 Ktx/sec | 7.5 Ktx/sec | | stdb_raw | 🧠 | person | unique | 0 | 100 | 160.7±0.19µs | 167.4±0.28µs | 6.1 Ktx/sec | 5.8 Ktx/sec | | stdb_raw | 🧠 | person | unique | 1000 | 100 | 165.8±0.16µs | 173.2±0.31µs | 5.9 Ktx/sec | 5.6 Ktx/sec | ## Full table iterate | db | on disk | schema | index type | new latency | old latency | new throughput | old throughput | |-------------|---------|----------|------------|---------------|---------------|----------------|----------------| | sqlite | 💿 | location | unique | - | 9.3±0.05µs | - | 104.7 Ktx/sec | | sqlite | 💿 | person | unique | - | 9.5±0.08µs | - | 102.6 Ktx/sec | | sqlite | 🧠 | location | unique | - | 7.8±0.07µs | - | 124.5 Ktx/sec | | sqlite | 🧠 | person | unique | - | 8.2±0.08µs | - | 119.4 Ktx/sec | | stdb_module | 💿 | location | unique | 53.5±7.36µs | 47.0±5.16µs | 18.3 Ktx/sec | 20.8 Ktx/sec | | stdb_module | 💿 | person | unique | 57.4±5.00µs | 54.7±2.99µs | 17.0 Ktx/sec | 17.9 Ktx/sec | | stdb_module | 🧠 | location | unique | 57.7±5.03µs | 61.1±5.22µs | 16.9 Ktx/sec | 16.0 Ktx/sec | | stdb_module | 🧠 | person | unique | 55.2±2.44µs | 55.9±1.81µs | 17.7 Ktx/sec | 17.5 Ktx/sec | | stdb_raw | 💿 | location | unique | 7.4±0.01µs | 7.7±0.01µs | 132.7 Ktx/sec | 126.2 Ktx/sec | | stdb_raw | 💿 | person | unique | 8.2±0.00µs | 8.6±0.00µs | 119.3 Ktx/sec | 114.2 Ktx/sec | | stdb_raw | 🧠 | location | unique | 7.3±0.00µs | 7.7±0.00µs | 133.7 Ktx/sec | 127.2 Ktx/sec | | stdb_raw | 🧠 | person | unique | 8.1±0.00µs | 8.5±0.00µs | 120.1 Ktx/sec | 114.9 Ktx/sec | ## Find unique key | db | on disk | key type | load | new latency | old latency | new throughput | old throughput | |-------------|---------|----------|------|----------------|----------------|----------------|----------------| | sqlite | 💿 | u32 | 1000 | - | 2.3±0.00µs | - | 424.1 Ktx/sec | | sqlite | 🧠 | u32 | 1000 | - | 1076.6±2.14ns | - | 907.0 Ktx/sec | | stdb_module | 💿 | u32 | 1000 | 27.5±1.75µs | 27.5±1.56µs | 35.5 Ktx/sec | 35.5 Ktx/sec | | stdb_module | 🧠 | u32 | 1000 | 25.6±1.10µs | 25.7±2.61µs | 38.1 Ktx/sec | 37.9 Ktx/sec | | stdb_raw | 💿 | u32 | 1000 | 1393.1±1.81ns | 1406.3±2.30ns | 701.0 Ktx/sec | 694.4 Ktx/sec | | stdb_raw | 🧠 | u32 | 1000 | 1322.3±0.91ns | 1334.7±1.35ns | 738.5 Ktx/sec | 731.7 Ktx/sec | ## Filter | db | on disk | key type | index strategy | load | count | new latency | old latency | new throughput | old throughput | |-------------|---------|----------|----------------|------|-------|----------------|----------------|----------------|----------------| | sqlite | 💿 | string | indexed | 1000 | 10 | - | 5.7±0.01µs | - | 171.6 Ktx/sec | | sqlite | 💿 | string | non_indexed | 1000 | 10 | - | 52.6±0.08µs | - | 18.6 Ktx/sec | | sqlite | 💿 | u64 | indexed | 1000 | 10 | - | 5.2±0.01µs | - | 187.9 Ktx/sec | | sqlite | 💿 | u64 | non_indexed | 1000 | 10 | - | 33.8±0.24µs | - | 28.9 Ktx/sec | | sqlite | 🧠 | string | indexed | 1000 | 10 | - | 4.3±0.01µs | - | 228.1 Ktx/sec | | sqlite | 🧠 | string | non_indexed | 1000 | 10 | - | 50.7±0.27µs | - | 19.3 Ktx/sec | | sqlite | 🧠 | u64 | indexed | 1000 | 10 | - | 3.8±0.00µs | - | 255.6 Ktx/sec | | sqlite | 🧠 | u64 | non_indexed | 1000 | 10 | - | 32.3±0.12µs | - | 30.2 Ktx/sec | | stdb_module | 💿 | string | indexed | 1000 | 10 | 32.3±1.79µs | 34.0±2.72µs | 30.2 Ktx/sec | 28.7 Ktx/sec | | stdb_module | 💿 | string | non_indexed | 1000 | 10 | 93.1±2.05µs | 94.3±3.45µs | 10.5 Ktx/sec | 10.4 Ktx/sec | | stdb_module | 💿 | u64 | indexed | 1000 | 10 | 31.2±1.26µs | 30.3±0.85µs | 31.3 Ktx/sec | 32.3 Ktx/sec | | stdb_module | 💿 | u64 | non_indexed | 1000 | 10 | 71.2±0.10µs | 78.1±4.40µs | 13.7 Ktx/sec | 12.5 Ktx/sec | | stdb_module | 🧠 | string | indexed | 1000 | 10 | 31.7±0.59µs | 33.2±2.11µs | 30.8 Ktx/sec | 29.4 Ktx/sec | | stdb_module | 🧠 | string | non_indexed | 1000 | 10 | 94.1±0.20µs | 93.1±1.74µs | 10.4 Ktx/sec | 10.5 Ktx/sec | | stdb_module | 🧠 | u64 | indexed | 1000 | 10 | 30.2±1.64µs | 29.0±2.11µs | 32.3 Ktx/sec | 33.7 Ktx/sec | | stdb_module | 🧠 | u64 | non_indexed | 1000 | 10 | 75.6±2.86µs | 78.5±3.56µs | 12.9 Ktx/sec | 12.4 Ktx/sec | | stdb_raw | 💿 | string | indexed | 1000 | 10 | 1655.5±6.69ns | 1678.3±2.09ns | 589.9 Ktx/sec | 581.9 Ktx/sec | | stdb_raw | 💿 | string | non_indexed | 1000 | 10 | 65.9±0.02µs | 71.3±0.07µs | 14.8 Ktx/sec | 13.7 Ktx/sec | | stdb_raw | 💿 | u64 | indexed | 1000 | 10 | 1583.1±1.05ns | 1606.0±3.26ns | 616.9 Ktx/sec | 608.1 Ktx/sec | | stdb_raw | 💿 | u64 | non_indexed | 1000 | 10 | 46.8±0.06µs | 55.0±0.01µs | 20.9 Ktx/sec | 17.8 Ktx/sec | | stdb_raw | 🧠 | string | indexed | 1000 | 10 | 1588.2±1.53ns | 1608.6±0.82ns | 614.9 Ktx/sec | 607.1 Ktx/sec | | stdb_raw | 🧠 | string | non_indexed | 1000 | 10 | 65.6±0.01µs | 71.3±0.03µs | 14.9 Ktx/sec | 13.7 Ktx/sec | | stdb_raw | 🧠 | u64 | indexed | 1000 | 10 | 1509.5±1.06ns | 1530.5±0.72ns | 647.0 Ktx/sec | 638.1 Ktx/sec | | stdb_raw | 🧠 | u64 | non_indexed | 1000 | 10 | 46.7±0.03µs | 54.8±0.08µs | 20.9 Ktx/sec | 17.8 Ktx/sec | ## Serialize | schema | format | count | new latency | old latency | new throughput | old throughput | |----------|---------------|-------|-----------------|-----------------|----------------|----------------| | location | bsatn | 100 | 1718.9±31.28ns | 1832.9±35.48ns | 55.5 Mtx/sec | 52.0 Mtx/sec | | location | json | 100 | 3.3±0.05µs | 3.1±0.02µs | 29.2 Mtx/sec | 30.6 Mtx/sec | | location | product_value | 100 | 561.3±0.72ns | 560.5±1.17ns | 169.9 Mtx/sec | 170.2 Mtx/sec | | person | bsatn | 100 | 2.4±0.00µs | 2.5±0.00µs | 39.6 Mtx/sec | 38.4 Mtx/sec | | person | json | 100 | 5.1±0.03µs | 4.8±0.01µs | 18.7 Mtx/sec | 19.7 Mtx/sec | | person | product_value | 100 | 651.7±1.04ns | 652.2±0.40ns | 146.3 Mtx/sec | 146.2 Mtx/sec | ## Module: invoke with large arguments | arg size | new latency | old latency | new throughput | old throughput | |----------|---------------|---------------|----------------|----------------| | 64KiB | 56.9±8.62µs | 65.0±4.12µs | - | - | ## Module: print bulk | line count | new latency | old latency | new throughput | old throughput | |------------|-------------------|-------------------|----------------|----------------| | 1 | 26.9±2.12µs | 27.9±2.09µs | - | - | | 100 | 199.9±4.17µs | 197.0±0.70µs | - | - | | 1000 | 1952.6±819.58µs | 1888.3±575.63µs | - | - | ## Remaining benchmarks | name | new latency | old latency | new throughput | old throughput | |------------------------------------------------------------------|-------------------|-----------------|----------------|----------------| | sqlite/💿/update_bulk/location/unique/load=1000/count=10 | - | 8.8±0.04µs | - | 110.8 Ktx/sec | | sqlite/💿/update_bulk/location/unique/load=1000/count=100 | - | 20.5±0.07µs | - | 47.7 Ktx/sec | | sqlite/💿/update_bulk/person/unique/load=1000/count=10 | - | 9.0±0.03µs | - | 109.1 Ktx/sec | | sqlite/💿/update_bulk/person/unique/load=1000/count=100 | - | 21.7±0.07µs | - | 44.9 Ktx/sec | | sqlite/🧠/update_bulk/location/unique/load=1000/count=10 | - | 3.6±0.01µs | - | 270.8 Ktx/sec | | sqlite/🧠/update_bulk/location/unique/load=1000/count=100 | - | 15.2±0.04µs | - | 64.3 Ktx/sec | | sqlite/🧠/update_bulk/person/unique/load=1000/count=10 | - | 3.7±0.02µs | - | 264.0 Ktx/sec | | sqlite/🧠/update_bulk/person/unique/load=1000/count=100 | - | 16.3±0.10µs | - | 59.8 Ktx/sec | | stdb_module/💿/update_bulk/location/unique/load=1000/count=10 | 383.8±3.26µs | 383.0±1.22µs | 2.5 Ktx/sec | 2.5 Ktx/sec | | stdb_module/💿/update_bulk/location/unique/load=1000/count=100 | 1056.6±14.06µs | 656.5±0.70µs | 946 tx/sec | 1523 tx/sec | | stdb_module/💿/update_bulk/person/unique/load=1000/count=10 | 446.3±1.62µs | 454.0±0.92µs | 2.2 Ktx/sec | 2.2 Ktx/sec | | stdb_module/💿/update_bulk/person/unique/load=1000/count=100 | 1054.4±542.86µs | 911.6±48.91µs | 948 tx/sec | 1097 tx/sec | | stdb_module/🧠/update_bulk/location/unique/load=1000/count=10 | 255.6±43.85µs | 222.9±0.15µs | 3.8 Ktx/sec | 4.4 Ktx/sec | | stdb_module/🧠/update_bulk/location/unique/load=1000/count=100 | 569.4±0.87µs | 566.5±7.37µs | 1756 tx/sec | 1765 tx/sec | | stdb_module/🧠/update_bulk/person/unique/load=1000/count=10 | 348.7±33.12µs | 401.2±1.95µs | 2.8 Ktx/sec | 2.4 Ktx/sec | | stdb_module/🧠/update_bulk/person/unique/load=1000/count=100 | 1127.9±6.71µs | 945.2±84.81µs | 886 tx/sec | 1057 tx/sec | | stdb_raw/💿/update_bulk/location/unique/load=1000/count=10 | 39.6±0.04µs | 39.5±0.04µs | 24.7 Ktx/sec | 24.7 Ktx/sec | | stdb_raw/💿/update_bulk/location/unique/load=1000/count=100 | 314.2±0.39µs | 317.0±0.61µs | 3.1 Ktx/sec | 3.1 Ktx/sec | | stdb_raw/💿/update_bulk/person/unique/load=1000/count=10 | 74.2±61.09µs | 55.6±0.11µs | 13.2 Ktx/sec | 17.6 Ktx/sec | | stdb_raw/💿/update_bulk/person/unique/load=1000/count=100 | 483.6±68.98µs | 469.6±0.41µs | 2.0 Ktx/sec | 2.1 Ktx/sec | | stdb_raw/🧠/update_bulk/location/unique/load=1000/count=10 | 27.9±0.03µs | 28.0±0.04µs | 35.0 Ktx/sec | 34.8 Ktx/sec | | stdb_raw/🧠/update_bulk/location/unique/load=1000/count=100 | 237.2±0.48µs | 241.3±0.23µs | 4.1 Ktx/sec | 4.0 Ktx/sec | | stdb_raw/🧠/update_bulk/person/unique/load=1000/count=10 | 38.1±0.04µs | 38.8±0.06µs | 25.6 Ktx/sec | 25.2 Ktx/sec | | stdb_raw/🧠/update_bulk/person/unique/load=1000/count=100 | 331.3±0.17µs | 340.7±0.98µs | 2.9 Ktx/sec | 2.9 Ktx/sec |
gefjon commented 3 months ago

Benchmark results appear to be within noise. The noise is admittedly quite large, and I don't think we have any benchmarks that hit the subscription manager, but what's important is that this isn't a huge regression to query performance. The actual improvements will come later, when we combine this with #840 , do follow-ups and optimize MemTable.

cloutiertyler commented 3 months ago

I think this was potentially merged prematurely @gefjon and @Centril. This is going to cause a lot of churn for Mario and Joshua as they are working on this code right now. Given that this doesn't directly improve performance, adding additional burden on them to rebase doesn't seem to make sense to me.

cloutiertyler commented 3 months ago

I think I actually need to revert this one to let Mario and Joshua finish their work first.

cloutiertyler commented 3 months ago

See: https://github.com/clockworklabs/SpacetimeDB/pull/879#issuecomment-1958103682