search

elastic / apm-aws-lambda

A repository for the AWS Lambda extension and other lambda related tools and build scripts.
https://www.elastic.co/guide/en/apm/guide/current/monitoring-aws-lambda.html
Apache License 2.0
15 stars 27 forks source link

Make invocation timeouts visible in Elastic APM #118

Open AlexanderWert opened 2 years ago

AlexanderWert commented 2 years ago

When a lambda invocation times out, then in most cases the APM agents won't be able to finalize and send the corresponding transaction to the extension and APM server. As a consequence, lambda function invocations that time out are not visible in APM at all. However, seeing timed out invocations in APM is essential as it is an important indicator that something is wrong with a function or it is too slow.

The lambda runtime notifies external extensions with the shutdown event in case of a timeout. This mechanism could be used by the APM extension to generate and send an artificial transaction (with a failure state and additional information on the timeout). In this way APM would provide information on timed out invocations.

SHolzhauer commented 2 years ago

I have started to run into this and took a while for me to figure out what was happening. It would make the APM extension for Lambda a lot more useful if it would show when a lambda times-out.

simitt commented 2 years ago

@SHolzhauer thanks for your feedback on this issue. We have it on our radar, but might not be able to immediately prioritize it.

malibeg commented 1 year ago

My team needs this feature also, is there any progress?

AlexanderWert commented 1 year ago

@malibeg

This has been solved from the Lambda extension side with https://github.com/elastic/apm-aws-lambda/issues/315, but it still requires the corresponding APM agents to support that new mode.

The tracking issue for the agents is: https://github.com/elastic/apm/issues/753