search

DataDog / saluki

An experimental toolkit for building telemetry data planes in Rust.
Apache License 2.0
12 stars 2 forks source link

[APR-208] chore: add cgroups v2 metadata collector #208

Closed tobz closed 3 weeks ago

tobz commented 3 weeks ago

Context

This PR addresses #186 by adding a new metadata collector for cgroups v2.

This metadata collector scrapes all discovered control groups under the unified v2 hierarchy that match a simple naming heuristic. It feeds the metadata aggregator, similar to the containerd collector, with ancestry links, to be able to translate from cgroup controller inode to container ID.

We've done some work overall in terms of beefing up the feature detector to support configuring the new collector, as well as some logging cleanup and unification around string interning and so on.

Notes

This was tested locally on a Linux machine using cgroups v2 (Ubuntu 23.10), with a single Docker container running. It correctly traverses the unified v2 hierarchy and finds the control group for the container, extracting both the correct container ID and the associated controller inode.

Fixes #186.

pr-commenter[bot] commented 3 weeks ago

Regression Detector (DogStatsD)

Regression Detector Results

Run ID: 74792bb3-1279-4e10-94e4-cee43c0bc671

Baseline: 7.52.0 Comparison: 7.52.1

Performance changes are noted in the perf column of each table:

No significant changes in experiment optimization goals

Confidence level: 90.00% Effect size tolerance: |Δ mean %| ≥ 5.00%

There were no significant changes in experiment optimization goals at this confidence level and effect size tolerance.

Fine details of change detection per experiment

| perf | experiment | goal | Δ mean % | Δ mean % CI | links | |------|----------------------------------------------|--------------------|----------|----------------|-------| | ➖ | dsd_uds_100mb_3k_contexts_distributions_only | memory utilization | +0.84 | [+0.60, +1.08] | | | ➖ | dsd_uds_10mb_3k_contexts | ingress throughput | +0.04 | [+0.02, +0.07] | | | ➖ | dsd_uds_512kb_3k_contexts | ingress throughput | +0.02 | [-0.03, +0.06] | | | ➖ | dsd_uds_1mb_50k_contexts | ingress throughput | +0.02 | [-0.01, +0.04] | | | ➖ | dsd_uds_100mb_250k_contexts | ingress throughput | +0.01 | [-0.06, +0.07] | | | ➖ | dsd_uds_500mb_3k_contexts | ingress throughput | -0.00 | [-0.00, +0.00] | | | ➖ | dsd_uds_100mb_3k_contexts | ingress throughput | -0.00 | [-0.01, +0.01] | | | ➖ | dsd_uds_1mb_50k_contexts_memlimit | ingress throughput | -0.02 | [-0.05, +0.02] | | | ➖ | dsd_uds_1mb_3k_contexts | ingress throughput | -0.02 | [-0.07, +0.03] | |

Explanation

A regression test is an A/B test of target performance in a repeatable rig, where "performance" is measured as "comparison variant minus baseline variant" for an optimization goal (e.g., ingress throughput). Due to intrinsic variability in measuring that goal, we can only estimate its mean value for each experiment; we report uncertainty in that value as a 90.00% confidence interval denoted "Δ mean % CI". For each experiment, we decide whether a change in performance is a "regression" -- a change worth investigating further -- if all of the following criteria are true: 1. Its estimated |Δ mean %| ≥ 5.00%, indicating the change is big enough to merit a closer look. 2. Its 90.00% confidence interval "Δ mean % CI" does not contain zero, indicating that *if our statistical model is accurate*, there is at least a 90.00% chance there is a difference in performance between baseline and comparison variants. 3. Its configuration does not mark it "erratic".
pr-commenter[bot] commented 3 weeks ago

Regression Detector (Saluki)

Regression Detector Results

Run ID: f63fa960-6d5f-4c3a-947d-e9cad647947d

Baseline: 86587be635bafecfd6cc547f03291c732d79c06c Comparison: 4f8fbd8fbe77684a8bb3959184fed5b3067908cc

Performance changes are noted in the perf column of each table:

No significant changes in experiment optimization goals

Confidence level: 90.00% Effect size tolerance: |Δ mean %| ≥ 5.00%

There were no significant changes in experiment optimization goals at this confidence level and effect size tolerance.

Fine details of change detection per experiment

| perf | experiment | goal | Δ mean % | Δ mean % CI | links | |------|-------------------------------------------------|--------------------|----------|----------------|-------| | ➖ | dsd_uds_100mb_3k_contexts_distributions_only | memory utilization | +1.78 | [+1.63, +1.93] | | | ➖ | dsd_uds_1mb_50k_contexts_memlimit | ingress throughput | +1.46 | [-1.88, +4.80] | | | ➖ | dsd_uds_500mb_3k_contexts | ingress throughput | +0.90 | [+0.78, +1.01] | | | ➖ | dsd_uds_1mb_3k_contexts | ingress throughput | +0.07 | [-0.16, +0.31] | | | ➖ | dsd_uds_512kb_3k_contexts | ingress throughput | +0.07 | [-0.08, +0.21] | | | ➖ | dsd_uds_100mb_250k_contexts | ingress throughput | +0.04 | [-0.24, +0.32] | | | ➖ | dsd_uds_10mb_3k_contexts | ingress throughput | +0.01 | [-0.10, +0.11] | | | ➖ | dsd_uds_100mb_3k_contexts | ingress throughput | +0.00 | [-0.01, +0.02] | | | ➖ | dsd_uds_50mb_10k_contexts_no_inlining | ingress throughput | -0.00 | [-0.00, +0.00] | | | ➖ | dsd_uds_50mb_10k_contexts_no_inlining_no_allocs | ingress throughput | -0.01 | [-0.05, +0.03] | | | ➖ | dsd_uds_1mb_50k_contexts | ingress throughput | -0.01 | [-0.05, +0.02] | |

Explanation

A regression test is an A/B test of target performance in a repeatable rig, where "performance" is measured as "comparison variant minus baseline variant" for an optimization goal (e.g., ingress throughput). Due to intrinsic variability in measuring that goal, we can only estimate its mean value for each experiment; we report uncertainty in that value as a 90.00% confidence interval denoted "Δ mean % CI". For each experiment, we decide whether a change in performance is a "regression" -- a change worth investigating further -- if all of the following criteria are true: 1. Its estimated |Δ mean %| ≥ 5.00%, indicating the change is big enough to merit a closer look. 2. Its 90.00% confidence interval "Δ mean % CI" does not contain zero, indicating that *if our statistical model is accurate*, there is at least a 90.00% chance there is a difference in performance between baseline and comparison variants. 3. Its configuration does not mark it "erratic".
pr-commenter[bot] commented 3 weeks ago

Regression Detector Links

Experiment Result Links

experiment link(s)
dsd_uds_100mb_250k_contexts [Profiling (ADP)] [Profiling (DSD)] [SMP Dashboard]
dsd_uds_100mb_3k_contexts [Profiling (ADP)] [Profiling (DSD)] [SMP Dashboard]
dsd_uds_100mb_3k_contexts_distributions_only [Profiling (ADP)] [Profiling (DSD)] [SMP Dashboard]
dsd_uds_10mb_3k_contexts [Profiling (ADP)] [Profiling (DSD)] [SMP Dashboard]
dsd_uds_1mb_3k_contexts [Profiling (ADP)] [Profiling (DSD)] [SMP Dashboard]
dsd_uds_1mb_50k_contexts [Profiling (ADP)] [Profiling (DSD)] [SMP Dashboard]
dsd_uds_1mb_50k_contexts_memlimit [Profiling (ADP)] [Profiling (DSD)] [SMP Dashboard]
dsd_uds_500mb_3k_contexts [Profiling (ADP)] [Profiling (DSD)] [SMP Dashboard]
dsd_uds_512kb_3k_contexts [Profiling (ADP)] [Profiling (DSD)] [SMP Dashboard]
dsd_uds_50mb_10k_contexts_no_inlining (ADP only) [Profiling (ADP)] [SMP Dashboard]
dsd_uds_50mb_10k_contexts_no_inlining_no_allocs (ADP only) [Profiling (ADP)] [SMP Dashboard]