search

open-telemetry / opentelemetry-collector-contrib

Contrib repository for the OpenTelemetry Collector
https://opentelemetry.io
Apache License 2.0
3.1k stars 2.39k forks source link

[pkg/ottl] Move debug log into Statement.Execute #36456

Open TylerHelmuth opened 2 days ago

TylerHelmuth commented 2 days ago

Description

@tdarwin noticed that the routingconnector wasnt emitting ottl logs when debug logging was enabled. This turned out to be because we added the debug statement to StatementSequence.Execute but the routingconnector uses Statement.Execute, but it only works with single statements.

This PR moves the debug log into the Statement.Execute function so that more components benefit from the debug logs.

Testing

Tested locally using this config:

receivers:
  otlpjsonfile:
    include:
      - /Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json

connectors:
  routing:
    default_pipelines: [traces/a]
    table:
      - statement: route() where attributes["service.name"] == "checkpoint-test"
        pipelines: [traces/b]

processors:
  transform:
    error_mode: ignore
    trace_statements:
      - context: resource
        statements:
          - set(attributes["test"], "pass")

exporters:
  debug/a:
  debug/b:

service:
  telemetry:
    logs:
      level: debug
  pipelines:
    traces:
      receivers:
        - otlpjsonfile
      exporters:
        - routing
    traces/a:
      receivers:
        - routing
      exporters:
        - debug/a
    traces/b:
      receivers:
        - routing
      processors:
        - transform
      exporters:
        - debug/b

The collector spit out these logs:

❯ ./bin/otelcontribcol_darwin_arm64 --config ./local/config.yaml
2024-11-19T12:29:41.671-0700    info    service@v0.114.0/service.go:166 Setting up own telemetry...
2024-11-19T12:29:41.671-0700    info    telemetry/metrics.go:70 Serving metrics {"address": "localhost:8888", "metrics level": "Normal"}
2024-11-19T12:29:41.672-0700    info    builders/builders.go:26 Development component. May change in the future.        {"kind": "exporter", "data_type": "traces", "name": "debug/b"}
2024-11-19T12:29:41.672-0700    debug   builders/builders.go:24 Alpha component. May change in the future.      {"kind": "processor", "name": "transform", "pipeline": "traces/b"}
2024-11-19T12:29:41.672-0700    info    builders/builders.go:26 Development component. May change in the future.        {"kind": "exporter", "data_type": "traces", "name": "debug/a"}
2024-11-19T12:29:41.672-0700    debug   builders/builders.go:24 Alpha component. May change in the future.      {"kind": "connector", "name": "routing", "exporter_in_pipeline": "traces", "receiver_in_pipeline": "traces"}
2024-11-19T12:29:41.673-0700    debug   builders/builders.go:24 Alpha component. May change in the future.      {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces"}
2024-11-19T12:29:41.673-0700    info    service@v0.114.0/service.go:238 Starting otelcontribcol...      {"Version": "0.114.0-dev", "NumCPU": 10}
2024-11-19T12:29:41.673-0700    info    extensions/extensions.go:39     Starting extensions...
2024-11-19T12:29:41.673-0700    info    service@v0.114.0/service.go:261 Everything is ready. Begin running and processing data.
2024-11-19T12:29:41.875-0700    debug   fileconsumer/file.go:125        matched files   {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:41.875-0700    debug   fileconsumer/file.go:157        Consuming files {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:42.076-0700    debug   fileconsumer/file.go:125        matched files   {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:42.076-0700    debug   fileconsumer/file.go:157        Consuming files {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:42.275-0700    debug   fileconsumer/file.go:125        matched files   {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:42.275-0700    debug   fileconsumer/file.go:157        Consuming files {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:42.474-0700    debug   fileconsumer/file.go:125        matched files   {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:42.474-0700    debug   fileconsumer/file.go:157        Consuming files {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:42.675-0700    debug   fileconsumer/file.go:125        matched files   {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:42.675-0700    debug   fileconsumer/file.go:157        Consuming files {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:42.875-0700    debug   fileconsumer/file.go:125        matched files   {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:42.875-0700    debug   fileconsumer/file.go:157        Consuming files {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:43.076-0700    debug   fileconsumer/file.go:125        matched files   {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:43.076-0700    debug   fileconsumer/file.go:157        Consuming files {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:43.274-0700    debug   fileconsumer/file.go:125        matched files   {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:43.274-0700    debug   fileconsumer/file.go:157        Consuming files {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:43.475-0700    debug   fileconsumer/file.go:125        matched files   {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:43.475-0700    debug   fileconsumer/file.go:157        Consuming files {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:43.675-0700    debug   fileconsumer/file.go:125        matched files   {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:43.675-0700    debug   fileconsumer/file.go:157        Consuming files {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "paths": ["/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"]}
2024-11-19T12:29:44.474-0700    info    fileconsumer/file.go:265        Started watching file   {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "path": "/Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.json"}
2024-11-19T12:29:44.475-0700    debug   ottl@v0.114.0/parser.go:34      TransformContext after statement execution      {"kind": "connector", "name": "routing", "exporter_in_pipeline": "traces", "receiver_in_pipeline": "traces", "statement": "route() where attributes[\"service.name\"] == \"checkpoint-test\"", "condition matched": true, "TransformContext": {"resource": {"attributes": {"service.name": "checkpoint-test", "platform.env": "prd"}, "dropped_attribute_count": 0}, "cache": {}}}
2024-11-19T12:29:44.475-0700    debug   ottl@v0.114.0/parser.go:340     initial TransformContext before executing StatementSequence     {"kind": "processor", "name": "transform", "pipeline": "traces/b", "TransformContext": {"resource": {"attributes": {"service.name": "checkpoint-test", "platform.env": "prd"}, "dropped_attribute_count": 0}, "cache": {}}}
2024-11-19T12:29:44.475-0700    debug   ottl@v0.114.0/parser.go:34      TransformContext after statement execution      {"kind": "processor", "name": "transform", "pipeline": "traces/b", "statement": "set(attributes[\"test\"], \"pass\")", "condition matched": true, "TransformContext": {"resource": {"attributes": {"service.name": "checkpoint-test", "platform.env": "prd", "test": "pass"}, "dropped_attribute_count": 0}, "cache": {}}}
2024-11-19T12:29:44.475-0700    info    Traces  {"kind": "exporter", "data_type": "traces", "name": "debug/b", "resource spans": 1, "spans": 1}
^C2024-11-19T12:29:48.931-0700  info    otelcol@v0.114.0/collector.go:328       Received signal from OS {"signal": "interrupt"}
2024-11-19T12:29:48.931-0700    info    service@v0.114.0/service.go:303 Starting shutdown...
2024-11-19T12:29:48.931-0700    info    extensions/extensions.go:66     Stopping extensions...
2024-11-19T12:29:48.931-0700    info    service@v0.114.0/service.go:317 Shutdown complete.