Routing processor fails to drop routing attribute when using default exporter

[swiatekm]

Component(s)

processor/routing

What happened?

Description

The routing processor doesn't drop routing attributes if none of the routes match.

Steps to Reproduce

Modify an existing test to use the default exporter, for example:

func TestLogs_RoutingWorks_ResourceAttribute_DropsRoutingAttributeOnDefaultRoute(t *testing.T) {
    defaultExp := &mockLogsExporter{}
    lExp := &mockLogsExporter{}

    host := newMockHost(map[component.DataType]map[component.ID]component.Component{
        component.DataTypeLogs: {
            component.NewID("otlp"):              defaultExp,
            component.NewIDWithName("otlp", "2"): lExp,
        },
    })

    exp, err := newLogProcessor(noopTelemetrySettings, &Config{
        AttributeSource:              resourceAttributeSource,
        FromAttribute:                "X-Tenant",
        DropRoutingResourceAttribute: true,
        DefaultExporters:             []string{"otlp"},
        Table: []RoutingTableItem{
            {
                Value:     "acme",
                Exporters: []string{"otlp/2"},
            },
        },
    })
    require.NoError(t, err)

    require.NoError(t, exp.Start(context.Background(), host))

    l := plog.NewLogs()
    rm := l.ResourceLogs().AppendEmpty()
    rm.Resource().Attributes().PutStr("X-Tenant", "acme2")
    rm.Resource().Attributes().PutStr("attr", "acme")

    assert.NoError(t, exp.ConsumeLogs(context.Background(), l))
    logs := defaultExp.AllLogs()
    require.Len(t, logs, 1, "log should be routed to default exporter")
    require.Equal(t, 1, logs[0].ResourceLogs().Len())
    attrs := logs[0].ResourceLogs().At(0).Resource().Attributes()
    _, ok := attrs.Get("X-Tenant")
    assert.False(t, ok, "routing attribute should have been dropped")
    v, ok := attrs.Get("attr")
    assert.True(t, ok, "non routing attributes shouldn't be dropped")
    assert.Equal(t, "acme", v.Str())
}

Expected Result

Routing attribute should always be dropped, even when using the default exporter.

Actual Result

Routing attribute is only dropped when using one of the explicit routes.

Collector version

0.82.0

Environment information

No response

OpenTelemetry Collector configuration

No response

Log output

No response

Additional context

No response

[github-actions[bot]]

Pinging code owners:

• processor/routing: @jpkrohling

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[jpkrohling]

@swiatekm-sumo, would you like to open a PR addressing this one?

[swiatekm]

@swiatekm-sumo, would you like to open a PR addressing this one?

Sure, you can assign to me. I'm not completely sure what the solution should look like though, as currently deleting the routing attribute is done via an OTTL statement during route determination. Since there's no OTTL for the default path, I'd need to add an awkward special case.

Maybe we should add the ability to run OTTL on the default path the same way we allow it in the routes?

[jpkrohling]

I'm not sure I like that. Users can still use the transform/attribute processors to remove that, can't they?

[swiatekm]

I'm not sure I like that. Users can still use the transform/attribute processors to remove that, can't they?

They can when using the connector, although this makes the pipeline more complex for a relatively simple use case. But the processor requires that it be the last one in the pipeline, so no more transformations. This is why this option was added in the first place.

[jpkrohling]

Got it. Given that it's our intention to deprecate the processor in favor of the connector, I'm not sure I would implement this at this point.

[github-actions[bot]]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• processor/routing: @jpkrohling

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions[bot]]

This issue has been closed as inactive because it has been stale for 120 days with no activity.