Replace the "serverless" framework with AWS SAM

[mnapoli]

Bref is currently using the serverless framework internally to deploy lambdas. I want to explore whether using AWS SAM could be an alternative.

	Serverles	AWS SAM
Cross-provider	✅	❌ (limited to AWS)
Easy deployment from CLI	✅	✅
Deployment using a CloudFormation stack	✅	✅
Allow extra CloudFormation resources	✅	✅
Simpler syntax than CloudFormation	✅ (serverless plugins)	✅ (SAM resources)
Allow to run hooks before deployment	❌ (plugins, but runs in the project's directory so messes up the local project)	❓
Run preview locally	✅	✅
Run preview in Docker	❌ (not natively)	✅
Run API Gateway locally	❌	✅
Run Lambda API locally	❌	✅
Invoke locally with fake S3/SQS/... events	❌	✅
Integration with CodePipeline/CodeDeploy	❌	✅
Deployment strategies (blue/green, canary deployment…)	❌	✅
Integration with AWS SAR (serverless application repository)	❌	✅
Deployment of different stages	✅	❓ https://github.com/awslabs/aws-sam-cli/issues/814
Deployment of different stages to different accounts	✅	❓
Auto-creates a S3 bucket for packaging the function	✅	❌
Deploy multiple functions at once	✅	✅
Deploy multiple functions that use different languages	✅	✅
View logs in CLI	✅	✅
Exclude some directories when deploying	✅	❌

Other lines to come, feel free to comment to add questions.

This comes with 2 questions:

• should Bref only target AWS?

  I'm leaning on a very strong yes here considering how everything even today with serverless (which is generic) is strictly tied to AWS. AWS is also the leading FaaS provider today, and supporting other providers would be a huge effort. By targeting only one we can provide the best DX, features and performances.

• should Bref hide the details of the deployment tools, or should Bref integrates in the deployment tool?

  What I'm getting at is that Bref could be a "whole in one" solution, hiding serverless/SAM (so that we control the details and we can provide a very simple user experience). Or Bref could be the "PHP part" that fits inside serverless/SAM (so that we can benefit from everything those frameworks have to offer).

[nealio82]

One huge positive here is that you’ll be able to more accurately replicate the Lambda environment locally. I find with debugging that needing to deploy between changes really slows down the feedback loop.

With regard to ‘should we only target AWS?’; this is, as you’re no doubt aware, a tough question. AFAIK we can currently only run Bref on Lambda ATM anyway, so I doubt the current user base will be affected. However, what would that mean for the project if AWS natively supported PHP at some point in the future? Would we switch back to serverless?

[mnapoli]

Agreed with the feedback loop, honestly what they did in SAM-CLI is pretty amazing, it works really well.

what would that mean for the project if AWS natively supported PHP at some point in the future? Would we switch back to serverless?

I would love for PHP to be supported natively! Bref has 3 goals right now:

• PHP support through the JS shim
• easy deployment (partially done through serverless, could be SAM)
• framework integrations

Removing the 1st goal would only mean PHP gets faster, official support and less things to maintain.

The 2nd goal is covered by Bref because IMO serverless and SAM are not good enough right now. Both do not allow some sort of "hooks" to run e.g. composer install before deploying. Well with serverless that could be done through a plugin but it messes up the local project which is a deal breaker. And SAM is not easy to use, here is how you deploy the application:

sam package --template-file sam.yaml --s3-bucket test-sam-bref --output-template-file packaged.yaml
sam deploy --template-file packaged.yaml --stack-name test-sam-bref --capabilities CAPABILITY_IAM

So the "deployment" part in Bref is necessary IMO, especially to attract [PHP] people over to serverless hosting.

And finally the framework integrations is something that AWS would never provide AFAICT, so Bref has a reason to exist if only just for that.

[mnapoli]

Just discovered there is sam build (https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-cli-command-reference-sam-build.html).

It seems to be running npm install or pip depending on the files it finds, it would be great if we could run anything here (I'm thinking installing Composer dependencies, the PHP binary, etc.). That would mean we could use sam directly instead of having to run bref commands which would wrap sam.

[sanathkr]

@mnapoli I am one of the devs that built sam build and built many parts of SAM CLI :) First off great comparison table between Serverless Framework & SAM.

sam build is powered by this separate library called aws-lambda-builders. All it has is a list of workflows for any programming language (ex: Python) + dependency manager (ex: PIP) + application framework (ex: chalice) combination. This library is already built into lambci docker container so you can get high fidelity builds inside a container if you want for free.

If you add a workflow for your framework, it will be immediately available to sam build. It currently only supports Python+PIP workflow, but we are actively working with the community on adding support for other languages that can run on AWS Lambda. Our goal is to support a large collection of build capabilities (Capability = Programming Language + Dependency Manager + Application Framework) so that eventually you run sam build and things just work.

If you want, I am happy to work with you to add a PHP workflow for lambda-builders.

Thoughts?

[mnapoli]

@sanathkr this is awesome, thank you for your help! I'm looking at this right now.

[incoming-th]

Should only focus on AWS?

Well, I do my best to try to test and report issues I found, but this make me think sometime that we try to force a system to use something not made for. It is challenging and interesting, @mnapoli did a very good job here, but what about OpenWhisk? This is natively supporting PHP (they said) and also have affordable pricing list with free tier.

IBM Cloud Functions Pricing

If AWS one day support PHP in Lambda this will remove a lot of headache, but still Bref and serverless are way easier to use than AWS dashboard.

AWS SAM?

Well again this seems nice, but it looks like - I can be wrong - it is only for people who are looking for a tool to manage their fleet of lambdas. In the case of Bref we try to "run" a framework (for example Laravel - Lumen should also work by the way) to get the benefit of the serverless. So basically we have very few running Lambdas. Plus serverless did a really good job with the yaml config file.

I am still surprised that AWS Lambda does not support PHP yet, as it is already stateless and should be easier to implement.

[mnapoli]

@MickaelTH AWS announced two weeks ago an API to support any language, including PHP. See #100 for that.

As far as SAM goes it does help to deploy multiple lambdas, but it's mostly about the deployment process itself as well as local development tools. It can do basically everything that serveless does but with more features, except it's only for AWS.

[nealio82]

I'm running into a large blocker with SAM local - any routes other than / do not work, responding with {"message": "missing authentication token"} - although the /{proxy+} does work when deployed to Lambda.

Apparently this is a known issue in SAM local but I haven't yet seen / come across a fix for it. A workaround is that you can add each of your root prefixes into template.yaml, which then works both locally and on Lambda, but this really slows down the development cycle to have to add all your new routes into template.yaml, then restart the local API to be able to test a new controller / route.

Otherwise i think my local development cycle might have to be along the lines of:

1. develop using php built-in webserver
2. test on SAM local after adding all local route prefixes to template.yaml (maybe this can be automated, at least for Symfony by capturing the output of debug:router, perhaps by adding a make test-local runner to the Makefile?)
3. make deploy to send it to Lambda, changing template.yaml back to /{proxy+} instead of with having explicit route prefixes

[mnapoli]

That's not ideal at all indeed.

Just to be sure I understand: what you are saying is that using /{proxy+} doesn't work at all with SAM locally?

If that's the case that's maybe a bug we should report back to their issue tracker, maybe they will fix it quickly if it's that important?

[nealio82]

That's correct; although I forgot to mention above that it's only GET requests which are affected. POST works ok. I haven't tried any other http methods yet. Threads I found on Google seem to suggest other people have found the same thing.

I spent a good few hours last night trying to get anything other than / working locally, although /{proxy+} works fine when deployed up to AWS.

I was looking for a way to make a regex for the path: ... property but I'm not sure it's possible, and in any case the {proxy+} is supposed to act as a catch-all anyway!

For example, setting path: /{proxy+} resulted in the error for GET /api, /api/books, /api/books/[id] (using api-platform's example code of books / reviews https://api-platform.com/docs/distribution#bringing-your-own-model), but POST /api/books works.

Setting path: /api/{proxy+} allowed me to do GET /api/books and GET /api/books/[id] with no problem. I can't remember if this config worked or not for /api - I can check later.

Both /{proxy+} and /api/{proxy+} seem to work fine when deployed to the cloud.

I also wondered if there's a way of overriding the auth key for this route, but tired eyes and lack of experience with SAM local configuration meant it led me nowehere. I also don't necessarily want to start overriding deployment auth configs for the sake of making things work locally (although I'm undecided on how much of an issue this actually is, as the PHP application should be bringing its own security model anyway)...

(FYI, method is set to ANY for all of the examples above)

[nealio82]

FTR I think this is already in their bug tracker and triaged: https://github.com/awslabs/aws-sam-cli/issues/437