trentm
commented
1 year ago Some of the API Gateway request and response data is captured in the APM Agent's Lambda instrumentation, e.g. https://github.com/elastic/apm-agent-nodejs/blob/5d419876454681e23d0e5fe42641ef3f32ca8a00/lib/lambda.js#L141-L150 However, not all of it. And that info is in the tracing data, not in log records.
Note in that linked-to code that the variable name is pseudoReq. This is creating an object that looks enough like a http.IncomingMessage for the data extraction that the APM agent's handler is doing. I'm not sure if the convertReqRes handling in the current ecs-logging-nodejs formatters could work similarly.
And yah... supporting Fetch API objects eventually is a good idea.
In my case I am running Node in an AWS Lambda environment where there is not a Node server running. Request data is available as APIGatewayProxyEvent or APIGatewayEventRequestContextV2 and responses as APIGatewayProxyResult or APIGatewayProxyStructuredResultV2.
Ideally this context would be gathered automatically by the apm-aws-lambda extension and accessible by ecs-logging-nodejs via the agent. But from what I can tell this isn't the case. This library doesn't log any HTTP request/response data as part of it's APM integration. And I don't see that this API Gateway data is collected by the extension today, assuming it is even available from the AWS Lambda Logs/Telemetry APIs.
As a stopgap for an indeterminate period of time I am using AsyncLocalStorage and a Pino mixin to gather this data from my function handlers and include it in my logs following ECS conventions.
If #68 resolves with a namespace-based solution and APM support remains lacking I would like to continue logging HTTP details on top-level ECS fields for consistency with non-Lambda services. This seems possible if I could create compatible instances of
http.IncomingMessageandhttp.ServiceResponsebut Node's HTTP API seems excessively complicated for this case.Ideally I could explicitly provide the ECS fields, similar to what I do today and how existing req/res support works. Alternately support for Fetch API Request and Response objects.