Structured log body is being sent as textPayload without structure

[dashpole]

{"insertId": "upsyhzmdeu","jsonPayload": {"logger_name": "<redacted>","method": "log","levelValue": 900,"thread_id": 56,"message": "Error in stream connection (will retry): <redacted>: java.net.ConnectException: Failed to connect to <redacted>","@timestamp": "2024-08-23T14:35:45.502Z","thread_name": "<redacted>","level": "WARNING","class": "<redacted>ChannelImpl"},"resource": {"type": "k8s_container","labels": {"pod_name": "<redacted>-79f75cb-6h5hs","cluster_name": "<redacted>-us-east1","location": "us-east1","container_name": "test","project_id": "<redacted-project>","namespace_name": "<redacted>-410"}},"timestamp": "2024-08-23T14:35:45.502691132Z","severity": "WARNING","labels": {"k8s-pod/<redacted>_service_name": "test","k8s-pod/app_kubernetes_io/instance": "<redacted>","k8s-pod/<redacted>/canonical-revision": "latest","compute.googleapis.com/resource_name": "<redacted>-us-east1-<redacted>", "k8s-pod/app_kubernetes_io/name": "grpc","k8s-pod/service_istio_io/canonical-name": "grpc","k8s-pod/pod-template-hash": "575cb","k8s-pod/security_istio_io/tlsMode": "istio","k8s-pod/app_kubernetes_io/component": "test"},"logName": "projects/<redacted>/logs/stderr","receiveTimestamp": "2024-08-23T14:35:50.519860025Z"}

From a report on slack, this is not sent as a json payload using the googlecloud exporter for logs.

[dashpole]

The customer is sending OTLP from a java app, and is using flogger (with jul for formatting) as logging framework.

The json in question was coming in as a string, rather than a map.

They are able to transform to a map using:

transform:
    log_statements:
      - context: log
        statements:
          # Extract everything between Str( and )
          - 'set(body, ParseJSON(body))'

But then they get errors when the log doesn't have a json body.

@psx95 have you tried OTLP logging from a java app before? Do you know if there is a way to make sure the body is sent as a map instead of a string?

Alternatively, we could make our exporter attempt to parse strings as json, but that might come with a performance penalty.

[psx95]

No, I haven't tried OTLP logging before. I will have to look into this.