search

lambci / git-lambda-layer

A layer for AWS Lambda that allows your functions to use `git` and `ssh` binaries
MIT License
344 stars 40 forks source link

Layer does not appear to function in Lambda nodejs10.x #13

Closed ryanblock closed 5 years ago

ryanblock commented 5 years ago

Git layer fails in the new Lambda nodejs10.x container released this week.

Repro steps:

This isn't particular surprising considering the messaging AWS has been doing this week about the underlying library changes in Lambda nodejs10.x.

Reference:

Their announcement email:

Hello,

We are updating the AWS Lambda and AWS Lambda@Edge execution environment to include recent versions of Amazon Linux and software packages.

A majority of functions will seamlessly benefit from the enhancements in this update without requiring you to take any action. However, in rare cases package updates may introduce compatibility issues. Functions that contain libraries or application code compiled against very specific underlying OS packages, specifically those for openssl, glibc, or other system libraries, may potentially be impacted.

Starting May 14, 2019, you can test your functions with the new execution environment. From May 21, 2019, all new functions or updates of existing functions will use the new execution environment. Your existing functions will automatically migrate to using the new execution environment on June 11, 2019.

See the following blog post to learn more about the update including the timeline, testing guidelines, and how you can prepare[1].

[1] https://aws.amazon.com/blogs/compute/upcoming-updates-to-the-aws-lambda-execution-environment/

Sincerely, Amazon Web Services

Edit: per tweeters, ratcheting down urgency since AWS Linux 2 isn't being forced into other Lambda container images yet; re-scoped issue more specifically to nodejs10.x

mhart commented 5 years ago

Yeah, that email you've pasted and the first link there in your issue are about the changes to the existing (non-nodejs10.x) runtimes. This layer should continue to work in existing runtimes – I've tested it lightly – feel free to open a separate issue if you see any incompatibilities there.

In terms of working in the nodejs10.x and any newer runtimes, that's a TODO – I'll update here when I get around to building and bundling all the extra pieces needed.

ryanblock commented 5 years ago

Yeah, I think before our conversation on Twitter I was confused by their language about the new AMI vs Linux 2 end (which I took to be one and the same).

I’m guessing you may have played with the new AMI, if so any differences / breaking changes? Would you like me to open a second issue for tracking that?

mhart commented 5 years ago

Yeah, all docker-lambda images have been updated with the new AMI, and this layer worked with it last I tried. Probably worth giving the ssh integration a decent workout, but git itself over https seemed to be fine.

Haven't ported any production CI loads over yet though, that'll probably be the test.

Removed binaries and libraries are here: https://gist.github.com/mhart/94f18d9327f5a101e8c16c47b7283dcc

But there were also some upgrades which I haven't documented anywhere:

Again, I don't think any of these should affect this layer – the few libs it does link to (libcurl, libstdc++) should still be compatible.

ryanblock commented 5 years ago

Thanks for all this information and research @mhart, so great. 🙌🏻

Feels on a single git package that bundles all its own dependencies, and is compatible with both container envs? Based on some tests I ran fully bundling some other Linux utilities a while back, I'm guessing that would be a pretty massive filesize...

mhart commented 5 years ago

The Amazon Linux 2 version will definitely be larger – not really a fan of foisting that on all the other runtime users. Besides, it'll be really fiddly trying to make it backwards compatible with, say, the libc version on Amazon Linux 1, while also including libraries that don't exist on Amazon Linux 2 but that do exist on Amazon Linux 1.

mhart commented 5 years ago

I'm guessing you guys use this in begin to do your CI builds?

ryanblock commented 5 years ago

The Amazon Linux 2 version will definitely be larger – not really a fan of foisting that on all the other runtime users

Yeah, that's legit. Presumably future runtimes in Lambda will be on AWS Linux 2, but for now that'd equate to a hard fork for a single runtime.

We use it in Begin for CI, yes, albeit indirectly. We use lambda-git (basically the package version of your layer, and bundles your git archive) because layer support in @architect is still nascent (and, to a lesser extent, vendor specific).

ryanblock commented 5 years ago

Any thoughts here? (Anything we can do to help?)

mhart commented 5 years ago

Are you wanting to move Begin to the nodejs10.x runtime?

ryanblock commented 5 years ago

@mhart sorry, missed this message. Yes, def, we have some functions that are stranded on Node 8 without a Node 10 image-compatible version of this layer.

Also, with last week's node8.10 EOL announced, seems like a good time.

If from your perspective you no longer want to maintain this layer, happy to help (or fork). Mostly just looking to resolve. Thanks!

mhart commented 5 years ago

Fixed thanks to the launch of https://github.com/lambci/yumda

Created a new layer named git-lambda2 for nodejs10.x

ARN: arn:aws:lambda:<region>:553035198032:layer:git-lambda2:1

ryanblock commented 5 years ago

Yooo, yumda is great. Have been thinking about if/how one could automate package → layer creation at greater scale, is that part of where you want to take it?

mhart commented 5 years ago

Yeah have thought about that a lot.

I think at the moment layers aren't all that great to be built/published at scale – ie it wouldn't be trivial to have a layer for each yum package.

I think we'll need to wait and see where AWS take SAR and see if they start to build in better support for layers.

In the meantime, yumda will at least make it easier to create layers (and ppl can publish them if they want)