Should exporter configuration be nested or at top level?

[jack-berg]

Related to #10, but specifically for exporters. Exporters could be defined at the top level and referenced by name, or could be expressed as nested arguments to the processors / metric readers which ultimately use them.

Let's talk about this!

Option 1: Exporters live at top level and are referenced by name

Advantages:

• Built-in reusability of components. I.e. no need to use YAML anchors to avoid repeating yourself.
• Less nesting
• Similar to collector configuration

Disadvantages:

• Exporters are not consistent across signals. Need to do validation to make sure referenced exporter is compatible with signal.
• OTLP exporter is different across signals: OTLP metric exporter has options for configuring aggregation temporality and default histogram aggregation. These options don't apply to spans and logs - its a bit confusing / awkward to configure an exporter with these options for a signal that will ignore them.

Example:

exporters:
  otlp/exporterA:
    endpoint: http://localhost:4317
    temporality_preference: delta
  otlp/exporterB:
    endpoint: http://remote-host:4317
    headers:
      api-key: 12345

span_processors:
  - name: batch
    exporter: otlp/exporterA
metric_readers:
  - name: periodic
    exporter: otlp/exporterA
log_processors:
  - name: batch
    exporter: otlp/exporter

Option 2: Exporters are nested under the components that use them

Advantages:

• More aligned with programatic SDK configuration, where exporters aren't directly configurable. I.e. you don't configure TracerProvider with a SpanExporter, you configure TracerProvider with a BatchSpanProcessor which has an a SpanExporter.
• Enables strict typing of signal specific exporters. I.e can express schema which says that zipkin and jaeger exporter can only be associated with spans, and NOT with metrics and logs.

Disadvantages:

• More nesting
• Divergent from collector config
• Need to rely on YAML anchors to avoid repetition, which may be unfamiliar to some people.

Example:

otlp_exporterA_args: &otlpExporterAArgs
  endpoint: https://localhost:4317

span_processors:
  - name: batch
    exporter:
      name: otlp
      args: *otlpExporterAArgs
metric_readers:
  - name: periodic
    exporter:
      name: otlp
      args: *otlpExporterAArgs
        temporality_preference: delta
log_processors:
  - name: batch
    exporter:
      name: otlp
      args:
        endpoint: http://remote-host:4317
        headers:
          api-key: 12345

[srikanthccv]

its a bit confusing / awkward to configure an exporter with these options for a signal that will ignore them.

What do you think about introducing the signal type for the exporters config? Something like this

exporters:
  span:
    otlp/exporterA:
      endpoint: http://localhost:4317
    otlp/exporterB:
      endpoint: http://remote-host:4317
      headers:
        api-key: 12345
  metric:
    otlp/exporterC:
      endpoint: http://localhost:4317
      temporality_preference: delta
  log:
    otlp/exporterD:
      endpoint: http://localhost:4317

[jack-berg]

I think that could work if those sections are nested under the respective providers:

sdk:
  tracer_provider:
    exporters:
      otlp:
        endpoint: http://localhost:4317
    span_processors:
      - name: batch
         args:
           exporter: otlp
  meter_provider:
    exporters:
      otlp:
        endpoint: http://localhost:4317
        temporality_preference: delta
    metric_readers:
      - name: periodic
         args:
           exporter: otlp

I wonder if having exporters separate would cause confusion for users that don't understand that they need to be paired with a batch span processor / batch log record processor / periodic metric reader.

[srikanthccv]

Maybe for first-time users, but I think people who have some exposure the configuring the SDK would already know that processors and exporters are paired together.

[jack-berg]

Closing this as we've made a tentative decision in #17. Can reoopen to address this comment if needed.