Validate Connector Usage

[arielvalentin]

Is your feature request related to a problem? Please describe.

I expected the configuration validator to find pipeline issues related to invalid usage of connector components, however the collector fails to start due to invalid usage and exits.

Here is an example that defines a count connector and uses it as a receiver in a metrics pipeline:

---
receivers:
  otlp:
    protocols:
      grpc:
      http:
exporters:
  logging:
connectors:
  count:

service:
  pipelines:
    traces/in:
      receivers: [otlp]
      exporters: [logging]
    metrics/out:
      receivers: [count]
      exporters: [logging]

The config.yaml passes validation:

otel-validations(:|✔) %
🤘 docker run -v $(pwd)/config.yaml:/etc/otelcol-contrib/config.yaml ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-contrib:0.87.0 validate --config=/etc/otelcol-contrib/config.yaml

otel-validations(:|✔) %
🤘 echo $?
0

However upon using this configuration, the collector reports an error and exits:

otel-validations(:|✔) %
🤘 docker run -v $(pwd)/config.yaml:/etc/otelcol-contrib/config.yaml ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-contrib:0.87.0
2023-10-22T21:05:36.612Z    info    service@v0.87.0/telemetry.go:84 Setting up own telemetry...
2023-10-22T21:05:36.612Z    info    service@v0.87.0/telemetry.go:201    Serving Prometheus metrics  {"address": ":8888", "level": "Basic"}
Error: failed to build pipelines: connector "count" used as receiver in metrics pipeline but not used in any supported exporter pipeline
2023/10/22 21:05:36 collector server run finished with error: failed to build pipelines: connector "count" used as receiver in metrics pipeline but not used in any supported exporter pipeline

otel-validations(:|✔) %
🤘 echo $?
1

  Describe the solution you'd like

I expect the validator to return an error

[mx-psi]

cc @djaglowski

[djaglowski]

This problem is not specific to connectors. The validate command currently does not consider supported data types for any class of component.

That said, I agree there are additional configuration errors, such as this one, which we could surface within the command if we would try to instantiate (but not start) a service.

[ringerc]

I raised this in #8866 after missing this issue in a search (somehow).

As I noted in that issue, it'd also be extremely helpful if the connector error mentioned the specific pipeline name in its complaint, e.g logs/out instead of just logs, like this:

Error: failed to build pipelines: connector "forward/test" used as receiver in "logs/out" pipeline but not used in any supported exporter pipeline

Currently it doesn't name the actual pipeline, so in complex configurations it can be tricky to tell what exactly it's complaining about.