Feature Request: Compiled gems from local machine, without lambdagems

[thepry]

Hi. Do you think it's possible to make jets use precompiled gems from a deploying machine? For example if we put them into .jets/precompiled_gems/? In that case we can just precompile them with docker image of amazon linux, can't we?

Btw, could you add more documentation on how to host your own precompiled gems here http://rubyonjets.com/docs/lambdagems/ ? Right now it's not clear in what format and url gems should be available there.

[tongueroo]

Think this is a great idea. Thinking about how to approach this as I think it can be improved. The best experience for users is not having to deal with and manage this kind of server-related stuff, that's the point of serverless. Want to let this simmer on my brain for a bit, that usually helps.

Also, resolved your other issue in "Bug: query params are not passed to Rails." Will release shortly. 👍

The structure is in the AWS docs: https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html

[thepry]

@tongueroo Thanks 👍

[thepry]

@tongueroo How do you build gems in Lambda gems? Is it just gem install? I'm trying to build sqlite3 in amazon linux docker and host it on s3.

[timgluz]

Hi,

I encountered similar issue - my project includes native extensions and they dont exists on the Lambdagems and sadly the documentation page doesnt say how i should host my own binaries;

Just wondering, is it possible to add a command to build a deployable code with the docker?

The command could use lambci/lambda:build-ruby2.5 image to compile native extensions and dump all the dependencies into the vendor/bundle folder, which we could easily zip into the deployment-package;

This way we could deploy code also on OSx and Windows machines and use private binary dependencies;

example of using docker to build dependencies for AWS Lambda:

docker run --rm -v "$PWD":/var/task lambci/lambda:build-ruby2.5 bundle install --path=vendor/bundle

• Is it possible to extend the code_building command so it could use docker image?

or

• Could you update documentation about hosting own packages?

[tongueroo]

Dug into this. The docker build process seems good, but it only works for the simplest cases. Even for common gems like mysql, it doesn’t work. You’ve got to deal with the quirks of each gem’s libraries development headers and confirm each one individually works. When the libraries change, you have to figure it out all over again. It would be ongoing issues and maintenance. So it’s not as simple as it might seem.

A better solution is Custom Lambda Layers. You can add your own Custom Lambda Layer to a Jets application. It provides full control over the build process. Use Docker, Firecracker (the actually microVM that AWS lambda uses), EC2, or whatever to build the Lambda Layer. It’ll work with the specific libraries and quirks for your application. Jets adds the Lambda Layer to the functions. It’s a nice simple way of handling it. Think this is why AWS has chosen to do it this way.

Disclosure: Think that the Lambda Gems service is the most friendly manner to handle this for Jets user. It saves people time. Lambda Gems is a considerable amount of work which is why it will have a partially paid service model. Hopefully, it helps to support Jets development also. Lambda Gems is currently in beta. Over time, if Lambda Gems is supported and grows, think it will become rarer for people to have to use a Custom Layer, which is great for the Jets user experience. Lambda Gems cannot handle every case though. That’s why there are Custom Lambda Layers also. Eventually, think will also add general build hooks so users can call out to whatever build process they wish.

[timgluz]

Awesome, didnt know about layers, they seems to be perfect solution for vendored packages;

I will give a try;