[pkg/stanza] container: remove time field from attributes after parsing

ChrsMark commented 3 months ago

Component(s)

pkg/stanza

Is your feature request related to a problem? Please describe.

This issue keeps track of what was suggested at https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/33353#discussion_r1626018076.

At the moment when the container parser parses the logs, it parses the time field moving the result to the Timestamp. However the original time field is preserved in the Attributes. Example:

2024-06-04T06:40:08.219Z    info    ResourceLog #0
Resource SchemaURL: 
Resource attributes:
     -> k8s.pod.uid: Str(d5ecc924-e255-4525-b5be-6437939b1e4d)
     -> k8s.container.name: Str(busybox)
     -> k8s.namespace.name: Str(default)
     -> k8s.pod.name: Str(daemonset-logs-dhzcq)
     -> k8s.container.restart_count: Str(0)
ScopeLogs #0
ScopeLogs SchemaURL: 
InstrumentationScope  
LogRecord #0
ObservedTimestamp: 2024-06-04 06:40:08.007370503 +0000 UTC
Timestamp: 2024-06-04 06:40:07.855932421 +0000 UTC
SeverityText: 
SeverityNumber: Unspecified(0)
Body: Str(otel logs at 06:40:07)
Attributes:
     -> logtag: Str(F)
     -> log.file.path: Str(/var/log/pods/default_daemonset-logs-dhzcq_d5ecc924-e255-4525-b5be-6437939b1e4d/busybox/0.log)
     -> time: Str(2024-06-04T06:40:07.855932421Z)
     -> log.iostream: Str(stdout)
Trace ID: 
Span ID: 
Flags: 0
LogRecord #1
ObservedTimestamp: 2024-06-04 06:40:08.007451031 +0000 UTC
Timestamp: 2024-06-04 06:40:07.957875321 +0000 UTC
SeverityText: 
SeverityNumber: Unspecified(0)
Body: Str(otel logs at 06:40:07)
Attributes:
     -> log.file.path: Str(/var/log/pods/default_daemonset-logs-dhzcq_d5ecc924-e255-4525-b5be-6437939b1e4d/busybox/0.log)
     -> log.iostream: Str(stdout)
     -> time: Str(2024-06-04T06:40:07.957875321Z)
     -> logtag: Str(F)
Trace ID: 
Span ID: 
Flags: 0

Describe the solution you'd like

As mentioned at https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/33353#discussion_r1626018076, it be would reasonable to remove the original field once its parsed.

One thing to consider here is the possibility of this field being used by some users today. This could happen for users that use the existing Helm preset. Since the container parser will be a replacement for this preset we need to take this into account. Maybe we can do this removal through a setting/feature-flag to not aggressively break any users that might use the original time field.

Describe alternatives you've considered

No response

Additional context

No response

github-actions[bot] commented 3 months ago

Pinging code owners:

pkg/stanza: @djaglowski

See Adding Labels via Comments if you do not have permissions to add labels yourself.

ChrsMark commented 2 months ago

Since https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/33946 was merged with the feature gate, let's keep this issue open to track the graduation of this feature gate. I will take care of it.

github-actions[bot] commented 1 week ago

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

pkg/stanza: @djaglowski

See Adding Labels via Comments if you do not have permissions to add labels yourself.

open-telemetry / opentelemetry-collector-contrib