[receiver/kafka]: Replace "topic" setting by "traces_topic", "logs_topic" and "metrics_topic"

[wildum]

Component(s)

receiver/kafka

Is your feature request related to a problem? Please describe.

When the setting "topic" is not specified, the same kafka receiver config can be used in all three pipelines if the topic names match the default values:

receivers:
  kafka:

pipelines: 
  metrics:
     receivers: [kafka] # consumes topic otlp_metrics
  logs:
     receivers: [kafka] # consumes topic otlp_logs
  traces:
     receivers: [kafka] # consumes topic otlp_spans

If the topic is set to any value, this structure will not work:

receivers:
  kafka:
    topic: custom_traces_topic

pipelines: 
  metrics:
     receivers: [kafka] # consumes topic custom_traces_topic => THIS WON'T WORK
  logs:
     receivers: [kafka] # consumes topic custom_traces_topic => THIS WON'T WORK
  traces:
     receivers: [kafka] # consumes topic custom_traces_topic

What happens in this case is that the three receivers will try to claim the same topic. This is a race condition that will succeed in 1/3 of scenarios.

To avoid this problem, the user must create three different configs for each pipeline. This is inconsistent with the default behavior that allows having one config for all three pipelines if the topic names are matching.

Describe the solution you'd like

The otlp receiver provides traces_url_path, metrics_url_path, and logs_url_path configuration to allow the URL paths that signal data needs to be sent to be modified per signal type.

The same principle can be applied to the topics of the kafka receiver:

receivers:
  kafka:
    traces_topic: custom_traces_topic # default otlp_spans
    metrics_topic: custom_metrics_topic # default otlp_metrics
    logs_topic: custom_logs_topic # default otlp_logs

pipelines: 
  metrics:
     receivers: [kafka] # consumes topic custom_metrics_topic
  logs:
     receivers: [kafka] # consumes topic custom_logs_topic
  traces:
     receivers: [kafka] # consumes topic custom_traces_topic

Describe alternatives you've considered

No response

Additional context

No response

[github-actions[bot]]

Pinging code owners:

• receiver/kafka: @pavolloffay @MovieStoreGuy

  See Adding Labels via Comments if you do not have permissions to add labels yourself.

[ChrsMark]

Wouldn't defining 3 different receivers work here?

receivers:
  kafka/metrics:
    topic: custom_metrics_topic
  kafka/logs:
    topic: custom_logs_topic
  kafka/traces:
    topic: custom_traces_topic

pipelines: 
  metrics:
     receivers: [kafka/metrics] 
  logs:
     receivers: [kafka/logs] 
  traces:
     receivers: [kafka/traces]

[wildum]

This works but this is inconsistent with the default behavior which allows using one receiver for three topics if you name your topics accordingly.

[ChrsMark]

Yeap, I think it makes sense to have more granular configuration options per signal. /cc @pavolloffay @MovieStoreGuy