newrelic / newrelic-lambda-extension

An AWS Lambda Extension to collect, enhance and transport telemetry to New Relic for AWS Lambda functions without requiring an external transport such as CloudWatch Logs or Kinesis.
Apache License 2.0
74 stars 52 forks source link

PHP Support #126

Open sc0ttdav3y opened 2 years ago

sc0ttdav3y commented 2 years ago

Summary

NewRelic has had a long history of supporting PHP, however I note this Lambda extension has no PHP support.

PHP already runs on Lambda quite well with the Bref (https://bref.sh) project. Bref has a NewRelic layer to bundle the php extension, but it currently requires a separate daemon server running elsewhere to ingest and forward logs. Your project seems to solve that problem by handling it locally in the Lambda function itself.

Desired Behavior

I'd like to be able to use Bref's NewRelic layer to install the NewRelic extension, and then use this layer to handles log ingestion and forwarding to NewRelic.

The desired result is a Lambda function that runs PHP and reports to NewRelic in a supported way, without needing any additional EC2 hosts configured to handle logging.

Possible Solution

Alternatives

I'm happy to help out here, but I don't currently understand how this lambda extension works differently to the newrelic daemon approach used by the PHP extension. If I can get some pointers to documentation on that or some tips, I'll be happy to run with it and contribute something if I'm successful.

Thanks!

mrickard commented 2 years ago

@sc0ttdav3y Thank you for the report! The extension's primarily a delivery mechanism for telemetry gathered by the agent and for logs. Agents typically have a different behavior in serverless vs non-serverless environments, due to the short invocation durations that are more common in serverless applications--in serverless mode, they send telemetry either to the extension for delivery, or to CloudWatch, assuming there's a log subscription set up to consume the logs and send the telemetry via the newrelic-log-ingestion Lambda.

I'll check with the PHP Agent team about what's needed, and point them to this issue.

coreyarnold commented 2 years ago

@elucus is the Product Manger overseeing our lambda product. I'll let him weigh in on this feature request.

sc0ttdav3y commented 2 years ago

Here's another attempt at doing a similar thing — this linked issue starts up the newrelic-daemon during the lambda extension lifecycle, and then points PHP's extension to send its data to localhost:31339.

starred-gijs commented 1 year ago

Hi @coreyarnold and @elucus, im the guy that attempted the Lambda Extension API idea to start the PHP daemon. I abandoned the idea because I could not get the connection stable enough, but that was due a lack of proper understanding how the daemon works and how to write the extension code. As a POC it succeeded, and I would be very interested to see approach be officially supported.

Currently we run the ECS docker image just for the PHP daemon, so a real serverless approach would be a big step up, also in terms of scalability and maintainability. If you need any feedback feel free to ping me.

sc0ttdav3y commented 1 year ago

@starred-gijs It was an up-hill battle, but I ended up getting success running OpenTelemetry on Lambda to instrument my PHP workloads. I ended up using the Go Lambda extension (built and supported by AWS), which runs a collector as a daemon like you were trying.

I'm consuming the OT traces in AWS X-Ray, but also I understand there's a way to push OT traces to other providers. I know Datadog supports OT traces, and perhaps New Relic can ingest OT traces too. It might be a supported path forward...

Anyway, it was by no means easy to set up, but now I have good observability, and a benefit of X-Ray is I get traceability all the way through the AWS stack into my PHP code. Here's a gist with some info on how I got OT working, along with info on the Lambda layers I used in case it helps anyone.

I'll be keeping my eye on this ticket for future support, and may move back to New Relic in future if it comes.

starred-gijs commented 1 year ago

Thanks for the reply and the gist, could be interesting to look into, although I think it will need a lot of manual work...