awslabs / aws-lambda-rust-runtime

A Rust runtime for AWS Lambda
Apache License 2.0
3.29k stars 335 forks source link

Make root level tracing span (TracingLayer) optional #911

Closed rimutaka closed 1 month ago

rimutaka commented 1 month ago

📬 A demo implementation for the log clutter issue discussed in #672. It needs more work (docs, tests, better env var parsing)

✍️ Description of changes:

Reuse AWS_LAMBDA_LOG_FORMAT to specify if the root span should not be created. E.g. AWS_LAMBDA_LOG_FORMAT=no-span or AWS_LAMBDA_LOG_FORMAT=json,no-span

The TracingLayer is added by default and not added if AWS_LAMBDA_LOG_FORMAT contains no-span.

Impact

  1. the change is backward compatible
  2. nothing changes for existing implementations
  3. minimal performance penalty for reading an env var
  4. no code changes are needed to change the log format

This last point allows using different log formats in different environments without code changes.

Examples

Including the root span

 INFO Lambda runtime invoke{requestId="local-request-id" xrayTraceId="Root=0-00000000-000000000000000000000000;Parent=0000000000000000;Sampled=0;Lineage=00000000:0"}: With global default provider
 INFO Lambda runtime invoke{requestId="local-request-id" xrayTraceId="Root=0-00000000-000000000000000000000000;Parent=0000000000000000;Sampled=0;Lineage=00000000:0"}: Command received: echo
ERROR Lambda runtime invoke{requestId="local-request-id" xrayTraceId="Root=0-00000000-000000000000000000000000;Parent=0000000000000000;Sampled=0;Lineage=00000000:0"}: "Error"

No root span (the main goal of this PR)

 INFO With global default provider
 INFO Command received: echo
ERROR "Error"

Root span + JSON

{"level":"INFO","fields":{"message":"With global default provider"},"span":{"requestId":"local-request-id","xrayTraceId":"Root=0-00000000-000000000000000000000000;Parent=0000000000000000;Sampled=0;Lineage=00000000:0","name":"Lambda runtime invoke"},"spans":[{"requestId":"local-request-id","xrayTraceId":"Root=0-00000000-000000000000000000000000;Parent=0000000000000000;Sampled=0;Lineage=00000000:0","name":"Lambda runtime invoke"}]}
{"level":"INFO","fields":{"message":"Command received: echo"},"span":{"requestId":"local-request-id","xrayTraceId":"Root=0-00000000-000000000000000000000000;Parent=0000000000000000;Sampled=0;Lineage=00000000:0","name":"Lambda runtime invoke"},"spans":[{"requestId":"local-request-id","xrayTraceId":"Root=0-00000000-000000000000000000000000;Parent=0000000000000000;Sampled=0;Lineage=00000000:0","name":"Lambda runtime invoke"}]}
{"level":"ERROR","fields":{"message":"\"Error\""},"span":{"requestId":"local-request-id","xrayTraceId":"Root=0-00000000-000000000000000000000000;Parent=0000000000000000;Sampled=0;Lineage=00000000:0","name":"Lambda runtime invoke"},"spans":[{"requestId":"local-request-id","xrayTraceId":"Root=0-00000000-000000000000000000000000;Parent=0000000000000000;Sampled=0;Lineage=00000000:0","name":"Lambda runtime invoke"}]}

No root span + JSON

{"level":"INFO","fields":{"message":"With global default provider"}}
{"level":"INFO","fields":{"message":"Command received: echo"}}
{"level":"ERROR","fields":{"message":"\"Error\""}}

🔏 By submitting this pull request

rimutaka commented 1 month ago

@calavera , thank you. It is very embarrassing how I missed it. The solution was hiding in the plain sight. Sorry for wasting so much of your time on this.