Support for the transform processor

[bmbferreira]

Is your feature request related to a problem? Please describe.

I'm using fluentbit to forward the logs to the opentelemetry collector and I want to parse the json from the logs and then transform some of the fields to attributes using the transform procesor. However that processor doesn't seem to be supported by the AWS otel collector.

Describe the solution you'd like

I would like to have support for the transform processor to be able to do advanced transformations using the Opentelemetry Transformation Language.

Describe alternatives you've considered

Use the docker image from opentelemetry instead of using this one from AWS.

Additional context N/A

[mhausenblas]

Thanks for creating this feature request, certainly something we consider @bmbferreira. Please do note that at the moment we do not support logs GA in ADOT (see https://github.com/aws-observability/aws-otel-community/issues/11 for details).

[jangaraj]

I would like to have transform processor for my trace processing - I would like to rename service.name attribute if it is not in allowed set, e.g.:

transform:
  error_mode: ignore
  trace_statements:
    - context: resource
      statements:
        - set(attributes["service.name"], "not-whitelisted-service") where isMatch(["service-a","service-b"], attributes["service.name"])

[mhausenblas]

Thank you, @jangaraj

[puckpuck]

+1 to getting this processor added to ADOT. We are working with a customer for whom the lack of this processor is causing heartburn.

[bpholt]

FWIW, I think this is a dupe of #1869. Very much hope it is picked up soon!

[github-actions[bot]]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[MaGaudin]

is there any update on this? it will be very very useful to have it.

[github-actions[bot]]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[hodgiwabi]

+1 for this processor, in the context of transforming histogram metrics for Dynatrace

[github-actions[bot]]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[schmidt-galen-heb]

Bumping this issue again.

I'd like to use this processor to extract attributes from a log's Scope, which doesn't seem to be possible with the processors currently included in the ADOT collector, but seems like it would be trivial with this processor:

transform/datadog/logs:
  error_mode: ignore
  log_statements:
    - context: log
      statements:
        - set(attributes["logger.name"], instrumentation_scope.name)

[will-sargent-eero]

bump, this matters because the opentelemetry-shim adds links to every span and Honeycomb treats a link as an extra span in searches.

[RafaelGBAccenture]

This feature is essential for efficiently processing message content towards CloudWatch. Having the transform processor would enable us to parse and filter logs effectively, allowing for better data management and more accurate insights. This capability is critical for creating efficient log filters and improving overall system observability. Thank you for considering this request.

[pankdhnd]

+1 for this. Transform processor is useful not only for logs but also for traces. It provides functionality for traces that no other processor provides. Kindly consider this request to have transform processor support as a part of ADOT.

[pcesara]

+100 This would be a very welcome feature.

Another use case is to manage attributes in metrics data points. For example, taking Resource attributes and adding them as attributes in either Spans or Metrics data points.

[github-actions[bot]]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[will-sargent-eero]

No go stale please

On Sun, Oct 6, 2024 at 1:02 PM github-actions[bot] @.***> wrote:

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 30 days.

— Reply to this email directly, view it on GitHub https://github.com/aws-observability/aws-otel-collector/issues/2300#issuecomment-2395566471, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMUFROSGFPAKOOY3LQRF3ITZ2GJOBAVCNFSM6AAAAAA4BQNHVKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOJVGU3DMNBXGE . You are receiving this because you commented.Message ID: @.***>