search

epsagon / epsagon-go

Automated tracing library for Go 1.x ⚡️
https://epsagon.com
MIT License
28 stars 11 forks source link

`recover` causes lost stack trace #21

Closed MatejBalantic closed 4 years ago

MatejBalantic commented 4 years ago

When a runtime panic occurs within AWS lambda, the aws SDK uses runtime.Callers() to collect stack trace, which is then reported to CloudWatch and to the client.

However the actual stack is lost due to epsagon-go calling a recover(). I noticed that the SDK is saving the stack trace to the invokeInfo, however that information isn't included in the error that is being rethrown

In case of panics like runtime error: invalid memory address or nil pointer dereference, it is very hard to trace the source of the bug without proper stack.

I would expect to have access to this information from the client, and in the CloudWatch log.

Maybe one option to mitigate this issue would be to embed the stack trace into a new error that should wrap the originally panicked error. It would make it a very long error message though, so that is not perfect.

Another option might be to open a PR with aws-lambda-go repo to allow stack trace injection when errors are being recovered upstream?

henperez commented 4 years ago

Thx for reporting this @MatejBalantic - We will work on fixing this ASAP

MatejBalantic commented 4 years ago

Hi there, is there any progress on this issue?

We are about to go into production, and we'll sadly have to drop Epsagon if this isn't resolved before. It's impossible to make anything from logs right now as we always just see Epsagon stack trace.

henperez commented 4 years ago

Hi @MatejBalantic, We'll release a fix for this early next week, will notify you here once it's released.