Application Trace analytics "TraceId" vs "traceId"

[pseidel-kcf]

I started using the new-to-AWS OpenSearch Service version 1.3 today and was excited to use the new Application Analytics feature . Our application logs have "TraceId" (with capital T) fields with OpenTelemetry for correlation but I get an error when attempting to correlate logs to traces in Application Trace analytics.

No Trace Id found in the event.
Please make sure to add "traceId" field to the logs.
More info on [Trace Analytics](https://opensearch.org/docs/latest/observability-plugin/trace/index/)
More info on [Log Correlation](https://opentelemetry.io/docs/reference/specification/logs/overview/#log-correlation)

I didn't see any mention of casing or specific fields in the linked Trace Analytics OpenSearch docs. The Log Correlation docs on the Open Telemetry docs link to a data model and their field is "TraceId" (with capital T). However when I search this code base I see "traceId" (with lowercase t) https://github.com/opensearch-project/observability/blob/b5c680278f35c2c3a570aa98a563603d59291f65/dashboards-observability/common/constants/explorer.ts#L39 in the constants.

We send our traces through Data Prepper to OpenSearch and the trace ids are in the same format between logs and traces.

Here's a truncated example log.

{
  "@t": "2022-07-27T18:45:24.6404029Z",
  "@m": "HTTP \"GET\" \"/hc-site\" responded 200 in 0.7446 ms",
  "RequestMethod": "GET",
  "RequestPath": "/hc-site",
  "StatusCode": 200,
  "Elapsed": 0.744602,
  "SpanId": "bb46b7a1c1f45ca3",
  "TraceId": "361a14ed14d7fe4002c8a272c729ee3d",
  "ParentId": "0000000000000000"
}

Is it possible to support "TraceId" (with uppercase T) or custom trace id fields? Additionally is possible to clarify requirements of this feature in the documentation?

[arulselvan]

I think the custom trace id field will make more sense... in my current log event trace id comes under the fields property(one more level)

[joshuali925]

trace-analytics strictly follows this schema: https://github.com/opensearch-project/data-prepper/blob/04dd7bd18977294800cf4b77d7f01914def75f23/docs/schemas/trace-analytics/otel-v1-apm-span-index-template.md

I believe it's derived from the Open Telemetry standard but not exactly the same, currently we don't support their or custom schemas. Could you explain a bit more on your set up? Ideally data prepper should convert open telemetry data from otel collector into its schema, then send to opensearch.