open-telemetry / opentelemetry-collector-contrib

Contrib repository for the OpenTelemetry Collector
https://opentelemetry.io
Apache License 2.0
2.72k stars 2.16k forks source link

[exporter/loadbalancer] Refactor the metrics export benchmarks #33676

Open RichieSams opened 1 week ago

RichieSams commented 1 week ago

To also test different numbers of ResourceMetrics, ScopeMetrics, and Datapoints

Description:

We want to be able to check how changes affect loadbalancing performance. However, the previous benchmarks only varied the number of Metrics, not the number of ResourceMetrics, ScopeMetrics, or Datapoints. This change adds all of those as variable parameters, and creates a set of combinations to test.

Link to tracking Issue: 32513

Testing:

Example of the output from the new benchmarks:

Running tool: /usr/local/go/bin/go test -benchmem -run=^$ -bench ^BenchmarkConsumeMetrics$ github.com/open-telemetry/opentelemetry-collector-contrib/exporter/loadbalancingexporter

goos: linux
goarch: amd64
pkg: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/loadbalancingexporter
cpu: Intel(R) Xeon(R) CPU E5-2687W v4 @ 3.00GHz
BenchmarkConsumeMetrics/service/1E_1RM_1SM_100M-24              8316        154195 ns/op       84310 B/op       1630 allocs/op
BenchmarkConsumeMetrics/service/1E_1RM_1SM_500M-24              1981        595325 ns/op      419667 B/op       8030 allocs/op
BenchmarkConsumeMetrics/service/1E_1RM_1SM_1000M-24              739       1434037 ns/op      821894 B/op      16030 allocs/op
BenchmarkConsumeMetrics/service/1E_1RM_3SM_33M-24               5947        170018 ns/op       83690 B/op       1626 allocs/op
BenchmarkConsumeMetrics/service/1E_1RM_3SM_166M-24              1599        640499 ns/op      419272 B/op       8010 allocs/op
BenchmarkConsumeMetrics/service/1E_1RM_3SM_333M-24               657       1595211 ns/op      837953 B/op      16026 allocs/op
BenchmarkConsumeMetrics/service/1E_3RM_1SM_33M-24               8799        162140 ns/op       84503 B/op       1658 allocs/op
BenchmarkConsumeMetrics/service/1E_3RM_1SM_166M-24              1227        837257 ns/op      420115 B/op       8042 allocs/op
BenchmarkConsumeMetrics/service/1E_3RM_1SM_333M-24               692       1529031 ns/op      838752 B/op      16058 allocs/op
BenchmarkConsumeMetrics/service/1E_3RM_3SM_11M-24               7893        135443 ns/op       86209 B/op       1694 allocs/op
BenchmarkConsumeMetrics/service/1E_3RM_3SM_55M-24               1858        749247 ns/op      408812 B/op       8030 allocs/op
BenchmarkConsumeMetrics/service/1E_3RM_3SM_111M-24               880       1225396 ns/op      828115 B/op      16094 allocs/op
BenchmarkConsumeMetrics/service/5E_1RM_1SM_100M-24              9862        115032 ns/op       84310 B/op       1630 allocs/op
BenchmarkConsumeMetrics/service/5E_1RM_1SM_500M-24              1960        558488 ns/op      419670 B/op       8030 allocs/op
BenchmarkConsumeMetrics/service/5E_1RM_1SM_1000M-24              848       1405581 ns/op      821868 B/op      16030 allocs/op
BenchmarkConsumeMetrics/service/5E_1RM_3SM_33M-24               6792        168208 ns/op       83702 B/op       1626 allocs/op
BenchmarkConsumeMetrics/service/5E_1RM_3SM_166M-24              1969        680526 ns/op      419289 B/op       8010 allocs/op
BenchmarkConsumeMetrics/service/5E_1RM_3SM_333M-24              1004       1392388 ns/op      837935 B/op      16026 allocs/op
BenchmarkConsumeMetrics/service/5E_3RM_1SM_33M-24               9571        128435 ns/op       84656 B/op       1664 allocs/op
BenchmarkConsumeMetrics/service/5E_3RM_1SM_166M-24              1546        789705 ns/op      420226 B/op       8048 allocs/op
BenchmarkConsumeMetrics/service/5E_3RM_1SM_333M-24               825       1537456 ns/op      838878 B/op      16064 allocs/op
BenchmarkConsumeMetrics/service/5E_3RM_3SM_11M-24               6825        163186 ns/op       86549 B/op       1706 allocs/op
BenchmarkConsumeMetrics/service/5E_3RM_3SM_55M-24               1929        657924 ns/op      408918 B/op       8036 allocs/op
BenchmarkConsumeMetrics/service/5E_3RM_3SM_111M-24               950       1154278 ns/op      828244 B/op      16100 allocs/op
BenchmarkConsumeMetrics/service/10E_1RM_1SM_100M-24             8512        160384 ns/op       84311 B/op       1630 allocs/op
BenchmarkConsumeMetrics/service/10E_1RM_1SM_500M-24             1621        699378 ns/op      419710 B/op       8030 allocs/op
BenchmarkConsumeMetrics/service/10E_1RM_1SM_1000M-24            1036       1099293 ns/op      821803 B/op      16030 allocs/op
BenchmarkConsumeMetrics/service/10E_1RM_3SM_33M-24              8652        130067 ns/op       83678 B/op       1626 allocs/op
BenchmarkConsumeMetrics/service/10E_1RM_3SM_166M-24             2067        674521 ns/op      419284 B/op       8010 allocs/op
BenchmarkConsumeMetrics/service/10E_1RM_3SM_333M-24              808       1246198 ns/op      837887 B/op      16026 allocs/op
BenchmarkConsumeMetrics/service/10E_3RM_1SM_33M-24              8563        156645 ns/op       84828 B/op       1670 allocs/op
BenchmarkConsumeMetrics/service/10E_3RM_1SM_166M-24             2100        599585 ns/op      420185 B/op       8048 allocs/op
BenchmarkConsumeMetrics/service/10E_3RM_1SM_333M-24              720       1466939 ns/op      839022 B/op      16070 allocs/op
BenchmarkConsumeMetrics/service/10E_3RM_3SM_11M-24              8817        135700 ns/op       86540 B/op       1706 allocs/op
BenchmarkConsumeMetrics/service/10E_3RM_3SM_55M-24              1321        825115 ns/op      408976 B/op       8036 allocs/op
BenchmarkConsumeMetrics/service/10E_3RM_3SM_111M-24              738       1511813 ns/op      828428 B/op      16106 allocs/op
BenchmarkConsumeMetrics/resource/1E_1RM_1SM_100M-24             7755        160514 ns/op       84379 B/op       1633 allocs/op
BenchmarkConsumeMetrics/resource/1E_1RM_1SM_500M-24             2239        555310 ns/op      419724 B/op       8033 allocs/op
BenchmarkConsumeMetrics/resource/1E_1RM_1SM_1000M-24             837       1245648 ns/op      821867 B/op      16033 allocs/op
BenchmarkConsumeMetrics/resource/1E_1RM_3SM_33M-24              7338        159220 ns/op       83760 B/op       1629 allocs/op
BenchmarkConsumeMetrics/resource/1E_1RM_3SM_166M-24             1262        853717 ns/op      419369 B/op       8013 allocs/op
BenchmarkConsumeMetrics/resource/1E_1RM_3SM_333M-24              945       1146094 ns/op      837925 B/op      16029 allocs/op
BenchmarkConsumeMetrics/resource/1E_3RM_1SM_33M-24              5770        181177 ns/op       84685 B/op       1667 allocs/op
BenchmarkConsumeMetrics/resource/1E_3RM_1SM_166M-24             1549        649104 ns/op      420273 B/op       8051 allocs/op
BenchmarkConsumeMetrics/resource/1E_3RM_1SM_333M-24              979       1391088 ns/op      838927 B/op      16067 allocs/op
BenchmarkConsumeMetrics/resource/1E_3RM_3SM_11M-24              9888        159606 ns/op       86399 B/op       1703 allocs/op
BenchmarkConsumeMetrics/resource/1E_3RM_3SM_55M-24              1897        580950 ns/op      408949 B/op       8039 allocs/op
BenchmarkConsumeMetrics/resource/1E_3RM_3SM_111M-24              932       1131839 ns/op      828278 B/op      16103 allocs/op
BenchmarkConsumeMetrics/resource/5E_1RM_1SM_100M-24             9764        143472 ns/op       84383 B/op       1633 allocs/op
BenchmarkConsumeMetrics/resource/5E_1RM_1SM_500M-24             1848        705359 ns/op      419777 B/op       8033 allocs/op
BenchmarkConsumeMetrics/resource/5E_1RM_1SM_1000M-24             799       1434908 ns/op      821940 B/op      16033 allocs/op
BenchmarkConsumeMetrics/resource/5E_1RM_3SM_33M-24              8358        144037 ns/op       83750 B/op       1629 allocs/op
BenchmarkConsumeMetrics/resource/5E_1RM_3SM_166M-24             1338        814019 ns/op      419358 B/op       8013 allocs/op
BenchmarkConsumeMetrics/resource/5E_1RM_3SM_333M-24              667       1597889 ns/op      838022 B/op      16029 allocs/op
BenchmarkConsumeMetrics/resource/5E_3RM_1SM_33M-24              8943        158059 ns/op       85008 B/op       1679 allocs/op
BenchmarkConsumeMetrics/resource/5E_3RM_1SM_166M-24             1654        672129 ns/op      420353 B/op       8057 allocs/op
BenchmarkConsumeMetrics/resource/5E_3RM_1SM_333M-24              772       1557139 ns/op      839164 B/op      16079 allocs/op
BenchmarkConsumeMetrics/resource/5E_3RM_3SM_11M-24              6537        180776 ns/op       86752 B/op       1715 allocs/op
BenchmarkConsumeMetrics/resource/5E_3RM_3SM_55M-24              1174        878871 ns/op      409266 B/op       8051 allocs/op
BenchmarkConsumeMetrics/resource/5E_3RM_3SM_111M-24              969       1140052 ns/op      828586 B/op      16115 allocs/op
BenchmarkConsumeMetrics/resource/10E_1RM_1SM_100M-24            6519        154740 ns/op       84373 B/op       1633 allocs/op
BenchmarkConsumeMetrics/resource/10E_1RM_1SM_500M-24            2044        557631 ns/op      419733 B/op       8033 allocs/op
BenchmarkConsumeMetrics/resource/10E_1RM_1SM_1000M-24            591       1712663 ns/op      821975 B/op      16033 allocs/op
BenchmarkConsumeMetrics/resource/10E_1RM_3SM_33M-24             8852        158954 ns/op       83767 B/op       1629 allocs/op
BenchmarkConsumeMetrics/resource/10E_1RM_3SM_166M-24            1380        749834 ns/op      419363 B/op       8013 allocs/op
BenchmarkConsumeMetrics/resource/10E_1RM_3SM_333M-24             628       1615069 ns/op      838072 B/op      16029 allocs/op
BenchmarkConsumeMetrics/resource/10E_3RM_1SM_33M-24             6498        175692 ns/op       85008 B/op       1679 allocs/op
BenchmarkConsumeMetrics/resource/10E_3RM_1SM_166M-24            1542        663767 ns/op      420502 B/op       8063 allocs/op
BenchmarkConsumeMetrics/resource/10E_3RM_1SM_333M-24             705       1489555 ns/op      839046 B/op      16073 allocs/op
BenchmarkConsumeMetrics/resource/10E_3RM_3SM_11M-24             8635        132854 ns/op       86736 B/op       1715 allocs/op
BenchmarkConsumeMetrics/resource/10E_3RM_3SM_55M-24             1669        726338 ns/op      409113 B/op       8045 allocs/op
BenchmarkConsumeMetrics/resource/10E_3RM_3SM_111M-24             880       1445868 ns/op      828689 B/op      16115 allocs/op
BenchmarkConsumeMetrics/metric/1E_1RM_1SM_100M-24               1773        597553 ns/op      134716 B/op       3112 allocs/op
BenchmarkConsumeMetrics/metric/1E_1RM_1SM_500M-24                253       4341556 ns/op      676283 B/op      15171 allocs/op
BenchmarkConsumeMetrics/metric/1E_1RM_1SM_1000M-24               105       9725885 ns/op     1373267 B/op      29854 allocs/op
BenchmarkConsumeMetrics/metric/1E_1RM_3SM_33M-24                2011        523729 ns/op      136888 B/op       3164 allocs/op
BenchmarkConsumeMetrics/metric/1E_1RM_3SM_166M-24                276       4016008 ns/op      715068 B/op      16285 allocs/op
BenchmarkConsumeMetrics/metric/1E_1RM_3SM_333M-24                114       9818961 ns/op     1479182 B/op      32926 allocs/op
BenchmarkConsumeMetrics/metric/1E_3RM_1SM_33M-24                2424        470926 ns/op      138238 B/op       3201 allocs/op
BenchmarkConsumeMetrics/metric/1E_3RM_1SM_166M-24                232       4350560 ns/op      728698 B/op      16630 allocs/op
BenchmarkConsumeMetrics/metric/1E_3RM_1SM_333M-24                100      10817668 ns/op     1535245 B/op      34199 allocs/op
BenchmarkConsumeMetrics/metric/1E_3RM_3SM_11M-24                2523        471705 ns/op      142912 B/op       3325 allocs/op
BenchmarkConsumeMetrics/metric/1E_3RM_3SM_55M-24                 337       3206410 ns/op      751098 B/op      17206 allocs/op
BenchmarkConsumeMetrics/metric/1E_3RM_3SM_111M-24                130       8071181 ns/op     1602315 B/op      35997 allocs/op
BenchmarkConsumeMetrics/metric/5E_1RM_1SM_100M-24               2492        442061 ns/op      136419 B/op       3192 allocs/op
BenchmarkConsumeMetrics/metric/5E_1RM_1SM_500M-24                397       2696737 ns/op      675084 B/op      15255 allocs/op
BenchmarkConsumeMetrics/metric/5E_1RM_1SM_1000M-24               214       5064444 ns/op     1376315 B/op      29910 allocs/op
BenchmarkConsumeMetrics/metric/5E_1RM_3SM_33M-24                2636        425315 ns/op      140510 B/op       3300 allocs/op
BenchmarkConsumeMetrics/metric/5E_1RM_3SM_166M-24                435       2562126 ns/op      714515 B/op      16397 allocs/op
BenchmarkConsumeMetrics/metric/5E_1RM_3SM_333M-24                165       6281069 ns/op     1491105 B/op      33291 allocs/op
BenchmarkConsumeMetrics/metric/5E_3RM_1SM_33M-24                2157        500601 ns/op      139394 B/op       3288 allocs/op
BenchmarkConsumeMetrics/metric/5E_3RM_1SM_166M-24                432       2811334 ns/op      732454 B/op      16778 allocs/op
BenchmarkConsumeMetrics/metric/5E_3RM_1SM_333M-24                204       5354481 ns/op     1531852 B/op      34216 allocs/op
BenchmarkConsumeMetrics/metric/5E_3RM_3SM_11M-24                2227        500970 ns/op      146582 B/op       3469 allocs/op
BenchmarkConsumeMetrics/metric/5E_3RM_3SM_55M-24                 313       3250037 ns/op      751348 B/op      17384 allocs/op
BenchmarkConsumeMetrics/metric/5E_3RM_3SM_111M-24                182       6877769 ns/op     1613246 B/op      36434 allocs/op
BenchmarkConsumeMetrics/metric/10E_1RM_1SM_100M-24              2822        425020 ns/op      139484 B/op       3257 allocs/op
BenchmarkConsumeMetrics/metric/10E_1RM_1SM_500M-24               513       2151195 ns/op      678636 B/op      15315 allocs/op
BenchmarkConsumeMetrics/metric/10E_1RM_1SM_1000M-24              225       4848433 ns/op     1378208 B/op      30036 allocs/op
BenchmarkConsumeMetrics/metric/10E_1RM_3SM_33M-24               1717        590405 ns/op      144393 B/op       3404 allocs/op
BenchmarkConsumeMetrics/metric/10E_1RM_3SM_166M-24               414       2510546 ns/op      722837 B/op      16650 allocs/op
BenchmarkConsumeMetrics/metric/10E_1RM_3SM_333M-24               219       4926953 ns/op     1500115 B/op      33520 allocs/op
BenchmarkConsumeMetrics/metric/10E_3RM_1SM_33M-24               2538        433734 ns/op      145807 B/op       3431 allocs/op
BenchmarkConsumeMetrics/metric/10E_3RM_1SM_166M-24               459       2258386 ns/op      743166 B/op      17063 allocs/op
BenchmarkConsumeMetrics/metric/10E_3RM_1SM_333M-24               229       5263669 ns/op     1545502 B/op      34552 allocs/op
BenchmarkConsumeMetrics/metric/10E_3RM_3SM_11M-24               2936        419635 ns/op      153338 B/op       3605 allocs/op
BenchmarkConsumeMetrics/metric/10E_3RM_3SM_55M-24                390       2582186 ns/op      767166 B/op      17769 allocs/op
BenchmarkConsumeMetrics/metric/10E_3RM_3SM_111M-24               178       5688235 ns/op     1620693 B/op      36720 allocs/op
PASS
ok      github.com/open-telemetry/opentelemetry-collector-contrib/exporter/loadbalancingexporter    252.011s

Documentation:

The benchmarks are documented inline in the code

RichieSams commented 1 week ago

Can someone add the "Skip changelog" label?

RichieSams commented 1 week ago

This PR is ready for final review