Disable system metrics by default in auto-instrumentation?

[srikanthccv]

The auto-instrumentation currently collects system metrics by default, which creates challenges in containerized environments. When pods are scaled, this behaviour results in unnecessary data duplication, dramatically increasing storage and processing costs without providing additional value. Moreover, in environments where multiple services (either in the same language or different language) run on the same host, having each service report system metrics is redundant and inefficient, especially when dedicated components like the OpenTelemetry Collector are already used to gather host metrics.

I propose disabling system metrics collection by default in OpenTelemetry Python auto-instrumentation. This change would significantly reduce costs, improve scalability in containerized workloads, and align better with best practices in complex, multi-service environments. It would encourage the use of dedicated collectors for system metrics, which is generally considered a more effective approach. Users who require system metrics from their Python services could still enable this feature explicitly, maintaining flexibility while improving the default behaviour.

[xrmx]

I have some code like this in the distro for being able to disable metrics not specific to the process:

    def load_instrumentor(self, entry_point: EntryPoint, **kwargs):
        ...
        instrumentor_kwargs = {}
        if instrumentor_class == SystemMetricsInstrumentor:
            if not system_metrics_enabled:
                instrumentor_kwargs["config"] = {
                    k: v for k, v in SYSTEM_METRICS_DEFAULT_CONFIG.items() if k.startswith("process.runtime")
                }
        instrumentor_class(**instrumentor_kwargs).instrument(**kwargs)