search

DataDog / datadog-lambda-python

The Datadog AWS Lambda Layer for Python
https://docs.datadoghq.com/integrations/amazon_lambda/#installing-and-using-the-datadog-layer
Apache License 2.0
84 stars 46 forks source link

[BUG] duration for APM aws.lambda.load span > cold_start parent span #354

Open pvicente opened 1 year ago

pvicente commented 1 year ago

Expected Behavior

aws.lambda.load span <= cold_start parent span

Actual Behavior

aws.lambda.load span > cold_start parent span

Steps to Reproduce the Problem

1. 1. 1.

Specifications

Serverless AWS lambda

datadog-lambda 4.75.0 dd_extension 45 dd_trace 1.7.1

Stacktrace

Child aws.lambda.load <= parent cold_start span

Screenshot 2023-07-27 at 15 54 14

I can see durations of 10 minutes or more in some traces

Screenshot 2023-07-27 at 15 58 45 
  Paste here
astuyve commented 1 year ago

hey @pvicente - thanks for reaching out! It seems that yes, there's a slight bug here where the load span is drawn across all modules - even when some modules are imported during the Lambda init process, and some are loaded during the first invocation.

Clearly this is most apparent when Lambda proactively initializes a sandbox (which we're finally allowed to discuss publicly).

We'll look into this and close this issue when we produce a fix.

Thank you!