siimon / prom-client

Prometheus client for node.js
Apache License 2.0
3.15k stars 378 forks source link

Improve performance of `hashObject` #594

Closed yosiat closed 1 year ago

yosiat commented 1 year ago

Since label names are predefined ahead-of-time and constant during runtime, we can sort them before-hand and use them when running hashObject.

Benchmarks

Running on node v18.18.2. Note: the registry benchmarks I couldn't run because of OOM (and on master branch as well).


http fetch GET 200 https://registry.npmjs.org/prom-client 83ms
http fetch GET 200 https://registry.npmjs.org/prom-client/-/prom-client-15.0.0.tgz 42ms
+ prom-client@15.0.0
added 4 packages from 4 contributors and audited 4 packages in 0.382s
found 0 vulnerabilities

- histogram ➭ observe#1 with 64 ➭ current x 104,381 ops/sec ±0.42% (99 runs sampled)
- histogram ➭ observe#1 with 64 ➭ prom-client@latest x 103,709 ops/sec ±0.72% (96 runs sampled)
- histogram ➭ observe#2 with 8 ➭ current x 68,178 ops/sec ±0.20% (98 runs sampled)
- histogram ➭ observe#2 with 8 ➭ prom-client@latest x 53,510 ops/sec ±0.18% (97 runs sampled)
- histogram ➭ observe#2 with 4 and 2 with 2 ➭ current x 34,169 ops/sec ±0.22% (97 runs sampled)
- histogram ➭ observe#2 with 4 and 2 with 2 ➭ prom-client@latest x 28,170 ops/sec ±0.32% (97 runs sampled)
- histogram ➭ observe#2 with 2 and 2 with 4 ➭ current x 36,057 ops/sec ±0.36% (97 runs sampled)
- histogram ➭ observe#2 with 2 and 2 with 4 ➭ prom-client@latest x 28,804 ops/sec ±0.20% (98 runs sampled)
- histogram ➭ observe#6 with 2 ➭ current x 23,360 ops/sec ±0.14% (97 runs sampled)
- histogram ➭ observe#6 with 2 ➭ prom-client@latest x 19,420 ops/sec ±0.36% (96 runs sampled)
- counter ➭ inc ➭ current x 59,643,643 ops/sec ±0.13% (101 runs sampled)
- counter ➭ inc ➭ prom-client@latest x 37,023,723 ops/sec ±0.46% (95 runs sampled)
- counter ➭ inc with labels ➭ current x 84,762 ops/sec ±0.25% (97 runs sampled)
- counter ➭ inc with labels ➭ prom-client@latest x 67,225 ops/sec ±0.62% (95 runs sampled)
- gauge ➭ inc ➭ current x 61,652,788 ops/sec ±0.41% (96 runs sampled)
- gauge ➭ inc ➭ prom-client@latest x 31,247,588 ops/sec ±0.28% (98 runs sampled)
- gauge ➭ inc with labels ➭ current x 86,552 ops/sec ±0.54% (97 runs sampled)
- gauge ➭ inc with labels ➭ prom-client@latest x 64,590 ops/sec ±0.47% (95 runs sampled)
- summary ➭ observe#1 with 64 ➭ current x 87,909 ops/sec ±0.18% (98 runs sampled)
- summary ➭ observe#1 with 64 ➭ prom-client@latest x 94,806 ops/sec ±0.87% (89 runs sampled)
- summary ➭ observe#2 with 8 ➭ current x 61,909 ops/sec ±0.15% (99 runs sampled)
- summary ➭ observe#2 with 8 ➭ prom-client@latest x 49,337 ops/sec ±0.35% (100 runs sampled)
- summary ➭ observe#2 with 4 and 2 with 2 ➭ current x 32,320 ops/sec ±0.20% (100 runs sampled)
- summary ➭ observe#2 with 4 and 2 with 2 ➭ prom-client@latest x 27,597 ops/sec ±0.79% (95 runs sampled)
- summary ➭ observe#2 with 2 and 2 with 4 ➭ current x 32,187 ops/sec ±0.21% (100 runs sampled)
- summary ➭ observe#2 with 2 and 2 with 4 ➭ prom-client@latest x 27,513 ops/sec ±0.37% (93 runs sampled)
- summary ➭ observe#6 with 2 ➭ current x 22,375 ops/sec ±0.47% (94 runs sampled)
- summary ➭ observe#6 with 2 ➭ prom-client@latest x 19,161 ops/sec ±0.14% (100 runs sampled)

┌───────────────────────────────┬────────────────────┬────────────────────┐
│ histogram                     │ current            │ prom-client@latest │
├───────────────────────────────┼────────────────────┼────────────────────┤
│ observe#1 with 64             │ 104381.0327549696  │ 103709.41386205897 │
├───────────────────────────────┼────────────────────┼────────────────────┤
│ observe#2 with 8              │ 68178.1543735591   │ 53509.94862568404  │
├───────────────────────────────┼────────────────────┼────────────────────┤
│ observe#2 with 4 and 2 with 2 │ 34169.376174278514 │ 28170.10920967872  │
├───────────────────────────────┼────────────────────┼────────────────────┤
│ observe#2 with 2 and 2 with 4 │ 36056.509189653516 │ 28804.04102513799  │
├───────────────────────────────┼────────────────────┼────────────────────┤
│ observe#6 with 2              │ 23359.579524196415 │ 19419.640968121184 │
└───────────────────────────────┴────────────────────┴────────────────────┘

┌─────────────────┬───────────────────┬────────────────────┐
│ counter         │ current           │ prom-client@latest │
├─────────────────┼───────────────────┼────────────────────┤
│ inc             │ 59643642.79575977 │ 37023723.28415886  │
├─────────────────┼───────────────────┼────────────────────┤
│ inc with labels │ 84761.87024308582 │ 67224.57758629428  │
└─────────────────┴───────────────────┴────────────────────┘

┌─────────────────┬───────────────────┬────────────────────┐
│ gauge           │ current           │ prom-client@latest │
├─────────────────┼───────────────────┼────────────────────┤
│ inc             │ 61652788.35302279 │ 31247587.722701166 │
├─────────────────┼───────────────────┼────────────────────┤
│ inc with labels │ 86551.57212390135 │ 64589.60038388497  │
└─────────────────┴───────────────────┴────────────────────┘

┌───────────────────────────────┬────────────────────┬────────────────────┐
│ summary                       │ current            │ prom-client@latest │
├───────────────────────────────┼────────────────────┼────────────────────┤
│ observe#1 with 64             │ 87908.71972414361  │ 94806.18075509806  │
├───────────────────────────────┼────────────────────┼────────────────────┤
│ observe#2 with 8              │ 61909.18982275139  │ 49336.623224028765 │
├───────────────────────────────┼────────────────────┼────────────────────┤
│ observe#2 with 4 and 2 with 2 │ 32319.8211562837   │ 27596.72566017053  │
├───────────────────────────────┼────────────────────┼────────────────────┤
│ observe#2 with 2 and 2 with 4 │ 32187.468085238048 │ 27513.002743185254 │
├───────────────────────────────┼────────────────────┼────────────────────┤
│ observe#6 with 2              │ 22375.38970678346  │ 19160.867264237862 │
└───────────────────────────────┴────────────────────┴────────────────────┘

✓ histogram ➭ observe#1 with 64 is 0.6476% faster.
✓ histogram ➭ observe#2 with 8 is 27.41% faster.
✓ histogram ➭ observe#2 with 4 and 2 with 2 is 21.30% faster.
✓ histogram ➭ observe#2 with 2 and 2 with 4 is 25.18% faster.
✓ histogram ➭ observe#6 with 2 is 20.29% faster.
✓ counter ➭ inc is 61.10% faster.
✓ counter ➭ inc with labels is 26.09% faster.
✓ gauge ➭ inc is 97.30% faster.
✓ gauge ➭ inc with labels is 34.00% faster.
⚠ summary ➭ observe#1 with 64 is 7.846% acceptably slower.
✓ summary ➭ observe#2 with 8 is 25.48% faster.
✓ summary ➭ observe#2 with 4 and 2 with 2 is 17.11% faster.
✓ summary ➭ observe#2 with 2 and 2 with 4 is 16.99% faster.
✓ summary ➭ observe#6 with 2 is 16.78% faster.
zbjornson commented 1 year ago

Thanks for the PR. This has been proposed before in #373 and would make it impossible to implement #298/#368. See also this comment about validly accepting subsets of labels: https://github.com/siimon/prom-client/pull/373#discussion_r426204193.

yosiat commented 1 year ago

@zbjornson thanks for the feedback.

First regarding the tests example in the comments:

const prom = require('.');

async function test1() {
    instance = new prom.Gauge({
        name: 'name_2',
        help: 'help',
        labelNames: ['code', 'success'],
    });
    instance.inc({ code: '200' }, 10);
    const str = await prom.Registry.globalRegistry.getSingleMetricAsString('name_2');
    console.log(str);
}

async function test2() {
    instance = new prom.Gauge({
        name: 'test',
        help: 'help',
        labelNames: ['code', 'success', 'ok'],
    });
    instance.inc({ code: '200' }, 10);
    instance.inc({ code: '200', ok: 'yes' }, 10);
    instance.inc({ ok: 'yes', success: 'false' }, 10);
    instance.inc({ success: 'false', ok: 'no' }, 10);
    const str = await prom.Registry.globalRegistry.getSingleMetricAsString('test');
    console.log(str);
}

async function main() {
    await test1();
    console.log("\n\n");
    await test2();
}

main();

I am getting this output:

# HELP name_2 help
# TYPE name_2 gauge
name_2{code="200"} 10

# HELP test help
# TYPE test gauge
test{code="200"} 10
test{code="200",ok="yes"} 10
test{ok="yes",success="false"} 10
test{success="false",ok="no"} 10

Which is the exact same output as master branch.

Regarding support of dynamic labels, I have strong opinion against this, since it can lead to a mess in metrics on large-scale codebases and having labels predefined gives more strictness. But, I don't think my PR is blocking having dynamic labels - see hashObject - if I don't get labelNames I fallback to extracting them like the old code.

Please re-consider this change, and I am happy to get more feedback here.

zbjornson commented 1 year ago

if I don't get labelNames I fallback to extracting them like the old code.

My bad, I missed the fallback. That's a great solution then.

Can you add a changelog entry please?

yosiat commented 1 year ago

@zbjornson added changelog entry, let me know if it's ok.

yosiat commented 1 year ago

@zbjornson @SimenB after addressing your questions & concerns, can we move forward with this PR? or there is something left here?

SimenB commented 11 months ago

https://github.com/siimon/prom-client/releases/tag/v15.1.0