search

softprops / lambda-rust

🐳 🦀 a dockerized lambda build env for rust applications
MIT License
162 stars 75 forks source link

Create an organization for all Rust lambda-related projects #99

Open zamazan4ik opened 3 years ago

zamazan4ik commented 3 years ago

Hi!

I've seen several projects from @softprops (btw - thanks for your work!) and seems like he has no enough time to maintain his projects, which are related to Rust serverless-related stack.

E,g, for this project we have a bunch of forks, which do completely the same thing - bump version, tune CI, etc.

I suggest to collaborate together and do the following things:

I am ready (and I am really interested in it) start this work: create an org, invite interested people, backport fixes, and so on.

What do you think about the idea?

I'll ping as many possible people from all issues in this and other repos. If I've forgotten someone - don't hesitate and mention them :) If you don't want to participate in it - sorry for the false mentioning :)

@ConfiguraAB @our-source @hans00 @jerusdp @juchiast @peterbean140 @westrik @NeilRobbins @kaicoh @tysg @micrictor

tysg commented 3 years ago

I'm up for it!

zamazan4ik commented 3 years ago

I want just wait a little bit more time and then start to work on it.

jerusdp commented 3 years ago

I'm putting a hand up too.

micrictor commented 3 years ago

I'm an AWS person 👋 - though not on a team relevant in any way to Rust Lambda functions.

I'm willing to take part in the organization in a personal capacity. In my personal opinion, it is unlikely that we'll get AWS to support a Serverless-framework specific build chain like this. They do support the base Lambda abstraction that anything using the default Serverless rust package imports.

Is there a technical reason that container builds are required or preferred to compile Rust code for AL2? Admittedly I've never done it in anything approaching a professional context, but it's caused me nothing but problems.

zamazan4ik commented 3 years ago

@micrictor Thanks for your interest! I've sent an invite to you.

At first, I just wanna clarify - here and below as Serverless I mean AWS Serverless infra (Lambda, etc.), not Serverless Framework. When we are talking about Serverless Framework, let's call it as Serverless Framework :) I just want to prevent some possible misunderstandings.

So, as far as I see, this repo has nothing special to Serverless Framework. The repo just provides a convenient Docker image for build Rust-based Lambda based on AWS AL2. Serverless Framework specific can be found here.

That's why I thought that AWS can be quite interested in maintaining such Docker image by their people since it improves AWS Lambda Rust users' experience, not Serverless Framework Rust users.

Is there a technical reason that container builds are required or preferred to compile Rust code for AL2? Admittedly I've never done it in anything approaching a professional context, but it's caused me nothing but problems.

That's a good question - I don't know the reason. Since I have no huge experience with Rust-based AWS lambdas, I don't know the answer. Maybe @softprops knows. If you have any information - please share it, if you can :)

By the way, new org is here: https://github.com/rust-serverless

I haven't done much work yet: registered new GMail and Docker Hub repo. Also tried to add Podman support for some scripts.

softprops commented 3 years ago

Hi folks just chiming in here.

It's true I have less and less time to curate and evolve these repos since I've become a father. I've just adjusted my priorities accordingly.

When I started with these, these there was no support for rust on lambda at all. Provided runtimes did not exist nor the now semi official rust one hosted by aws labs.

I've done what I could for a while and for a period that runtime went stale with no releases until just recently and things seem good again.

Since then I've learned a lot about making rust work on lambda and proper and improper use of lambda as well.

To answer the docker question. It was as almost required with the state of rust cross compilation and Amazon Linux 1 at the time when I first started. Amazon Linux 2 supersedes it's former and I suspect aws will be phasing out amazon Linux 1 on lambda so it would be a good idea to upgrade if you haven't already.

Rusts cross compile story improved as well over the years as the underlying aws Linux dist did so docker isn't really as needed as it used to be. I was actually considering removing support for docker to make maintaining this serverless framework plug-in simpler. There are some recipes for making OS X, Linux, and windows host envs build lambda compatible binaries in the code base now. I would just use those. Docker was a bit of crutch I leaned on and the actual image got more and more complex as I needed to support more and more use cases. Most of that should not be needed with the ability of installing the hat you need on your host machine

My own tooling preferences have shifted. At work and personal projects I almost use aws sam exclusively. I was a late adopter but I've grown to like it. Serverless framework the tool and the company have hinted at a few odd directions and it's been hard to follow along. Aws sam affords a lot of benefits albeit there is no plug-in model out side of make files atm. Where I'll likely be spending more of my time is improving the sam story for rust.

That said I'd been perfectly okay with donating this plug-in to a new fresh set of maintainers and the new org if y'all would like it it would help move it forward.

I myself still be working on improving the rust story for rust and serverless just at a different capacity and I'm different areas.

I do expect and see evidence that aws is firmly invested in rust now, both internally and externally. What I believe is that they are going to start filling some of the developer experience gaps I have been looking to fill in the past. This removes some of the burden of maintenance on the community and I'll look forward to that as an aws customer!

softprops commented 3 years ago

After typing all that out I realized that this is the docker repo and not the serverless framework plug-in repo :)

The same comments apply. I'd be happy to donate both to the new org

zamazan4ik commented 3 years ago

It's true I have less and less time to curate and evolve these repos since I've become a father. I've just adjusted my priorities accordingly.

My congratulations!

You did a great job and that's absolutely fine that your priorities are changed. Thanks to the open-source - we can continue your work.

My own tooling preferences have shifted. At work and personal projects I almost use aws sam exclusively. I was a late adopter but I've grown to like it. Serverless framework the tool and the company have hinted at a few odd directions and it's been hard to follow along.

It's a little bit offtop, I know. Can you explain please, from your point of view, what is wrong with a current direction of Serverless framework? That's really interesting for me since at my company we currently evaluating different approaches for going serverless on AWS.

Veetaha commented 3 years ago

It's true I have less and less time to curate and evolve these repos since I've become a father. I've just adjusted my priorities accordingly.

My congratulations!

You did a great job and that's absolutely fine that your priorities are changed. Thanks to the open-source - we can continue your work.

My own tooling preferences have shifted. At work and personal projects I almost use aws sam exclusively. I was a late adopter but I've grown to like it. Serverless framework the tool and the company have hinted at a few odd directions and it's been hard to follow along.

It's a little bit offtop, I know. Can you explain please, from your point of view, what is wrong with a current direction of Serverless framework? That's really interesting for me since at my company we currently evaluating different approaches for going serverless on AWS.

We in our company are completely against the serverless framwork. Its API is very inconvenient in the way how it requires leveraging yaml and cloudformation. Also, cloudformation itself if pretty crappy, it doesn't support new AWS services as they get released right away. We also find it very slow in deployment.

Instead of SAM, serverless framework and any other CloudFormation-based deployments we use terraform instead. We've also built a custom (unfortunately currently private) framework for building, deploying and testing (tests that deploy, teardown the resources, and tools for their garbage collection) in our company. We would be interested in open-sourcing all of those, it's just the question of time and our priorties to do all of that.

The question about using the docker image tp build lambda binaries is actually a good one. We've been using this docker image in our build framework for a while and it does have some problems like it can only build a single lambda per invocation resulting in huge build times in release where we disable incremental compilation.

It would be great if we could drop docker for building lambda binaries, @softprops if you have good guidance on how to do this, I would be greatful to hear

Veetaha commented 3 years ago

Also, @ZaMaZaN4iK it would be great if you could add me to the org. I'd also move https://github.com/softprops/dynomite there, because we are actively using it, so I'd like to have permissions to merge and release dynomite crate!

zamazan4ik commented 3 years ago

The question about using the docker image tp build lambda binaries is actually a good one. We've been using this docker image in our build framework for a while and it does have some problems like it can only build a single lambda per invocation resulting in huge build times in release where we disable incremental compilation.

There are several approaches. At least we can try to modify existing Docker image so it will be able to build 2 and more lambdas at once.

Regarding the question about avoiding Docker usage. E.g. you can try to prepare EC2 VM image on which provided.al2 is based and use it for building lambdas - the result should be the same. However, I don't think that it's really convenient way for building but it depends on your use cases.

micrictor commented 3 years ago

Isn’t one of the strengths of rust that cross-compiling is (supposed) to be straight forward? I’ll have to nuke my install to try to document the bootstrap for cross-compile.

Regardless, I agree with what you said about priorities. First, we get container builds into a good state, then we can worry about changing the build process.

On Sat, Aug 21, 2021 at 9:47 AM Alexander Zaitsev @.***> wrote:

The question about using the docker image tp build lambda binaries is actually a good one. We've been using this docker image in our build framework for a while and it does have some problems like it can only build a single lambda per invocation resulting in huge build times in release where we disable incremental compilation.

There are several approaches. At least we can try to modify existing Docker image so it will be able to build 2 and more lambdas at once.

Regarding the question about avoiding Docker usage. E.g. you can try to prepare EC2 VM image on which provided.al2 is based and use it for building lambdas - the result should be the same. However, I don't think that it's really convenient way for building but it depends on your use cases.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/softprops/lambda-rust/issues/99#issuecomment-903143366, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABTCAGINNBLQ35ABIEKDLJTT57KBVANCNFSM5A2MCSPQ .