Add support for collector confmap providers in the config

[rapphil]

If I try to use confmap providers in the collector configuration for receivers (and exporters), the Operator fails with the message: receivers property in the configuration doesn't contain valid receivers.

I think the operator should support confmap providers since they are part of the "core" collector configuration.

I'm providing the following unit test for config_validate_test.go to reproduce the problem:

func TestConfmapValidate(t *testing.T) {
    // prepare
    receivers := `
httpd/mtls:
  protocols:
    http:
      endpoint: mysite.local:55690
jaeger:
  protocols:
    grpc:
prometheus:
  protocols:
    grpc:
    `
    // First Test - Exporters
    http.HandleFunc("/config", func(w http.ResponseWriter, r *http.Request) {
        io.WriteString(w, receivers)
    })
    configStr := `
receivers: ${http://localhost:1234/config}

processors:

exporters:
  logging:

service:
  pipelines:
    metrics:
      receivers: [httpd/mtls, jaeger]
      exporters: [logging]
    metrics/1:
      receivers: [httpd/mtls, jaeger]
      exporters: [logging]
`
    // // prepare
    err := http.ListenAndServe(":1234", nil)
    config, err := ConfigFromString(configStr)
    require.NoError(t, err)
    require.NotEmpty(t, config)
    response, err := http.Get("http://localhost:1234/config")

    require.Nil(t, err)

    body, err := io.ReadAll(response.Body)

    require.Nil(t, err)

    require.Equal(t, receivers, string(body[:]))

    // test
    check := GetEnabledReceivers(logger, config)
    require.NotEmpty(t, check)
}

Suggestion: would be possible to use the same logic as the collector for validating/reading the configuration?

[frzifus]

Suggestion: would be possible to use the same logic as the collector for validating/reading the configuration?

What do you mean by the same logic? Using the collector libs might not work since people can specify custom collector images with e.g. older or unreleased collector versions.

I think one option would be to boot the collector and let the binary evaluate the given configuration. But I assume thats a topic in itself.

[pavolloffay]

At the moment we let users to specify the entire collector config in the config field of the CR. If there are certain features of the config that we do not support, we should at least document the limitations (or fix our implementation).

[rapphil]

or if the config cannot be parsed by the operator, maybe it should just issue an warning. If I understood correctly, the operator try to parse the configuration to look for ports that could be open etc. In case it cannot parse the configuration, it should just skip that.

[tekicat]

Operator doesn't understand when the receiver configs are mounted as files. and fails to create the required services for OTLP receivers

    receivers:
      prometheus: ${file:/etc/otelcol-contrib/prometheus-scrape-config.yaml}
      k8sobjects:
        auth_type: serviceAccount
        objects:
          - name: events
            mode: watch
            group: events.k8s.io
      otlp:
        protocols:
          grpc:
            endpoint: ${env:MY_POD_IP}:4317
          http:
            endpoint: ${env:MY_POD_IP}:4318