[8.x](backport #5767) Add support for running EDOT inside of running Elastic Agent

What does this PR do?

Adds the ability to run the EDOT along side the running Elastic Agent.

This connects the EDOT into the coordinator of the Elastic Agent. At any point if any of these top-level keys (receivers, processors, exporters, extensions, service) exist in the configuration or policy for the elastic-agent the EDOT is started. If all of those keys are removed from the configuration or policy then the EDOT is automatically stopped. If any configuration change occurs the updated configuration is passed along to the EDOT to handle.

Why is it important?

This allows EDOT configuration to exist inside of the configuration or policy and allow it to work as expected.

Checklist

[x] My code follows the style guidelines of this project
[x] I have commented my code, particularly in hard-to-understand areas
~~[ ] I have made corresponding changes to the documentation~~
[x] I have made corresponding change to the default configuration files
[x] I have added tests that prove my fix is effective or that my feature works
[x] I have added an entry in ./changelog/fragments using the changelog tool
[x] I have added an integration test or an E2E test

Disruptive User Impact

This is an addition and doesn't affect the way the current Elastic Agent runs at all.

How to test this PR locally

Place OTel configuration into the elastic-agent.yml:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318
processors:
  batch:

exporters:
  otlp:
    endpoint: otelcol:4317

extensions:
  health_check:
  pprof:

service:
  extensions: [health_check, pprof]
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp]
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp]
    logs:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp]

Run elastic-agent run -e.

Related issues

Closes #5796

This is an automatic backport of pull request #5767 done by Mergify.

Cherry-pick of b07566b96a95daa11897c7fbf0b9320618e2540b has failed:

On branch mergify/bp/8.x/pr-5767
Your branch is up to date with 'origin/8.x'.

You are currently cherry-picking commit b07566b96a.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
    modified:   .github/workflows/golangci-lint.yml
    new file:   changelog/fragments/1729011748-Add-EDOT-hybrid-mode.yaml
    modified:   control_v2.proto
    modified:   internal/pkg/agent/application/actions/handlers/handler_action_upgrade_test.go
    modified:   internal/pkg/agent/application/apm_config_modifier.go
    modified:   internal/pkg/agent/application/application.go
    modified:   internal/pkg/agent/application/config_test.go
    modified:   internal/pkg/agent/application/coordinator/coordinator.go
    modified:   internal/pkg/agent/application/coordinator/coordinator_state.go
    modified:   internal/pkg/agent/application/coordinator/coordinator_test.go
    modified:   internal/pkg/agent/application/coordinator/coordinator_unit_test.go
    modified:   internal/pkg/agent/application/coordinator/diagnostics_test.go
    modified:   internal/pkg/agent/application/gateway/fleet/fleet_gateway.go
    modified:   internal/pkg/agent/application/info/agent_id.go
    modified:   internal/pkg/agent/application/monitoring/liveness.go
    modified:   internal/pkg/agent/application/monitoring/liveness_test.go
    modified:   internal/pkg/agent/application/monitoring/reload/reload.go
    modified:   internal/pkg/agent/application/monitoring/v1_monitor.go
    modified:   internal/pkg/agent/application/upgrade/artifact/config.go
    modified:   internal/pkg/agent/application/upgrade/artifact/config_test.go
    modified:   internal/pkg/agent/application/upgrade/upgrade.go
    modified:   internal/pkg/agent/cmd/container.go
    modified:   internal/pkg/agent/cmd/container_test.go
    modified:   internal/pkg/agent/cmd/enroll_cmd.go
    modified:   internal/pkg/agent/cmd/enroll_cmd_test.go
    modified:   internal/pkg/agent/cmd/inspect.go
    modified:   internal/pkg/agent/cmd/otel.go
    modified:   internal/pkg/agent/cmd/otel_flags.go
    modified:   internal/pkg/agent/cmd/otel_flags_test.go
    deleted:    internal/pkg/agent/cmd/otel_windows.go
    deleted:    internal/pkg/agent/cmd/otel_windows_test.go
    modified:   internal/pkg/agent/cmd/status.go
    modified:   internal/pkg/agent/cmd/validate.go
    modified:   internal/pkg/agent/configuration/configuration.go
    modified:   internal/pkg/composable/controller.go
    modified:   internal/pkg/composable/providers/docker/docker.go
    modified:   internal/pkg/composable/providers/host/host.go
    modified:   internal/pkg/composable/providers/kubernetes/kubernetes.go
    modified:   internal/pkg/composable/providers/kubernetes/pod_test.go
    modified:   internal/pkg/composable/providers/kubernetesleaderelection/kubernetes_leaderelection.go
    modified:   internal/pkg/composable/providers/kubernetessecrets/kubernetes_secrets.go
    modified:   internal/pkg/composable/providers/local/local.go
    modified:   internal/pkg/composable/providers/localdynamic/localdynamic.go
    modified:   internal/pkg/config/config.go
    modified:   internal/pkg/config/config_test.go
    modified:   internal/pkg/config/loader.go
    modified:   internal/pkg/core/monitoring/config/config_test.go
    new file:   internal/pkg/otel/agentprovider/provider.go
    new file:   internal/pkg/otel/agentprovider/provider_test.go
    modified:   internal/pkg/otel/components.go
    deleted:    internal/pkg/otel/config_file_provider.go
    deleted:    internal/pkg/otel/config_file_provider_test.go
    deleted:    internal/pkg/otel/config_manager.go
    new file:   internal/pkg/otel/manager/extension.go
    new file:   internal/pkg/otel/manager/force_extension_converter.go
    new file:   internal/pkg/otel/manager/manager.go
    new file:   internal/pkg/otel/manager/manager_test.go
    new file:   internal/pkg/otel/otelhelpers/status.go
    new file:   internal/pkg/otel/otelhelpers/status_test.go
    modified:   internal/pkg/otel/run.go
    modified:   internal/pkg/otel/run_test.go
    modified:   internal/pkg/otel/validate.go
    modified:   internal/pkg/remote/client.go
    modified:   pkg/component/config.go
    modified:   pkg/control/v1/proto/control_v1.pb.go
    modified:   pkg/control/v1/proto/control_v1_grpc.pb.go
    modified:   pkg/control/v2/client/client.go
    modified:   pkg/control/v2/cproto/control_v2.pb.go
    modified:   pkg/control/v2/cproto/control_v2_grpc.pb.go
    modified:   pkg/control/v2/server/server.go
    modified:   pkg/control/v2/server/server_test.go
    modified:   pkg/features/features.go
    modified:   pkg/limits/limits.go
    modified:   pkg/testing/fixture.go
    modified:   testing/integration/diagnostics_test.go
    modified:   testing/integration/otel_test.go

Unmerged paths:
  (use "git add <file>..." to mark resolution)
    both modified:   NOTICE.txt
    both modified:   go.mod
    both modified:   go.sum

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

elastic / elastic-agent