[pkg/translator/azure] Unable to decode "malformed" data in Azure Resource Logs

[cparkins]

Component(s)

pkg/translator/azure, receiver/azureeventhub

What happened?

Description

We recently configured an Azure Event Hub Receiver to capture logs from an Azure API Management instance and noticed that the log data was lacking almost all of the data passed in.

Steps to Reproduce

Attempt to convert the following JSON event to OTel:

{
    "records": [
        {
            "DeploymentVersion": "0.40.16708.0",
            "Level": 4,
            "isRequestSuccess": true,
            "time": "2023-10-26T14:22:43.3416357Z",
            "operationName": "Microsoft.ApiManagement/GatewayLogs",
            "category": "GatewayLogs",
            "durationMs": 243,
            "callerIpAddress": "13.14.15.16",
            "correlationId": "REDACTED",
            "location": "East US",
            "properties": {
                "method": "GET",
                "url": "REDACTED",
                "backendResponseCode": 200,
                "responseCode": 200,
                "responseSize": 102945,
                "cache": "none",
                "backendTime": 54,
                "requestSize": 632,
                "apiId": "demo-api",
                "operationId": "GetSessions",
                "apimSubscriptionId": "master",
                "clientTime": 190,
                "clientProtocol": "HTTP/1.1",
                "backendProtocol": "HTTP/1.1",
                "apiRevision": "1",
                "clientTlsVersion": "1.2",
                "backendMethod": "GET",
                "backendUrl": "https://api.azurewebsites.net/sessions"
            },
            "resourceId": "/RESOURCE_ID",
            "resultType": "Succeeded",
            "truncated": 0
        }
    ]
}

Expected Result

Data is successfully Converted with all properties as accurate as possible.

Actual Result

Data is partially Converted, with very little data present (Essentially the timestamp and a few other details, no properties). See JSON Output below: {"resourceLogs":[{"resource":{"attributes":[{"key":"azure.resource.id","value":{"stringValue":"/RESOURCE_ID"}}]},"scopeLogs":[{"scope":{"name":"otelcol/azureresourcelogs","version":"0.88.0-dev"},"logRecords":[{"timeUnixNano":"1698343020000000000","body":{},"attributes":[{"key":"cloud.provider","value":{"stringValue":"azure"}},{"key":"azure.category","value":{"stringValue":""}},{"key":"azure.operation.name","value":{"stringValue":""}}],"traceId":"","spanId":""},{"timeUnixNano":"1698343080000000000","body":{},"attributes":[{"key":"azure.category","value":{"stringValue":""}},{"key":"azure.operation.name","value":{"stringValue":""}},{"key":"cloud.provider","value":{"stringValue":"azure"}}],"traceId":"","spanId":""}

Collector version

v0.88.0

Environment information

Environment

OS: Mac OS Compiler(if manually compiled): go 1.20.5 darwin/arm64

OpenTelemetry Collector configuration

receivers:
  azureeventhub:
    connection: "REDACTED"

service:
  telemetry:
    logs:
      level: "debug"

  pipelines:
    logs:
      receivers: [azureeventhub]
      processors: []
      exporters: [logging]

Log output

No error output seen.

Additional context

This appears to be an issue when the underlying JSON data can not be parsed correctly.

[github-actions[bot]]

Pinging code owners:

• pkg/translator/azure: @open-telemetry/collector-approvers @atoulme
• receiver/azureeventhub: @atoulme @djaglowski

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[crobert-1]

Fixed by #28650