search

aws-samples / aws-lambda-extensions

A collection of sample extensions to help you get started with AWS Lambda Extensions
MIT No Attribution
442 stars 145 forks source link

go logs extension api #79

Closed rpateloak9 closed 11 months ago

rpateloak9 commented 1 year ago

We are trying to use external extension to send our logs directly to S3 from our lambdas to step away from the pain of managing log groups when trying to send logs to a third party vendor.

I tried the python zip file approach first ( [s3-logs-extension-demo-zip-archive] ) but I soon realized that we need to package up the python interpreter along with this extension since our main function runs on .net 6. When i tried following the docs and I uploaded the lambda layer to our .net 6 functions, i would get an error indication "python not found"

then i tried the go-example-logs-api-extension since its cross platform binary and should run on any lambda runtime. The logs indicate the extension started up successfully, registered correctly, created the lambda bucket but no logs were ever written to s3. I checked the cloudtrail logs and permissions look good but it seems to me like the "logs data buffer might be empty even though there are logs." Any suggestion here?

are there any working extensions that i can take from here and run it on my lambdas that have different runtimes?

rpateloak9 commented 1 year ago

@julianwood btw thank you for your hard work with all this content, its is greatly appreciated.

julianwood commented 1 year ago

The go extensions should be working. Have you tried this one: https://github.com/aws-samples/aws-lambda-extensions/tree/main/go-example-telemetry-api-extension which pushes to a generic URL target or this one with adaptive batching for S3: https://github.com/aws-samples/aws-lambda-extensions/tree/main/go-example-adaptive-batching-extension

rpateloak9 commented 11 months ago

The go extensions should be working. Have you tried this one: https://github.com/aws-samples/aws-lambda-extensions/tree/main/go-example-telemetry-api-extension which pushes to a generic URL target or this one with adaptive batching for S3: https://github.com/aws-samples/aws-lambda-extensions/tree/main/go-example-adaptive-batching-extension

unfortunately we went with another route as extensions were leading to all sorts of issues for me and slowness. thanks