Deploying frontend via Amplify CLI

SimonEismann commented 4 years ago

I am currently trying to deploy this application via the Amplify CLI (I need a fully automated deployment).

I have tried checking out the git repo and running amplify init as well as amplify init --app https://github.com/aws-samples/aws-serverless-airline-booking. This deploys the backend fine. If I then add hosting and attempt to publish the frontend like this:

amplify hosting add
amplify publish

This results in the successful deployment of cloud resources and the console returning a hosting endpoint. However, then I get the following message:

npm ERR! code ENOENT
npm ERR! syscall open
npm ERR! path /app/package.json
npm ERR! errno -2
npm ERR! enoent ENOENT: no such file or directory, open '/app/package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2020-03-16T18_02_36_204Z-debug.log
frontend build command exited with code 254

/app is the base directory, that contains the project. I randomly checked some amplify samples and they all contain a package.json in the base folder. However, this is my first time using Amplify, so this might entirely be a usage error on my end.

heitorlessa commented 4 years ago

Hey @SimonEismann - Thanks for reaching out! This method won't work with this App but simpler front-ends/static websites.

Have you seen this doc? -> https://github.com/aws-samples/aws-serverless-airline-booking/blob/develop/docs/getting_started.md

You can use the 1-click Deploy, then use the pull command to bring that locally to you

SimonEismann commented 4 years ago

Thanks for the quick reply @heitorlessa

I have tried the 1-click deploy option and it works fine. However, I am looking to automate the deployment in order to run many repetitions of performance measurements. Do you have any suggestions for how to tackle this with your project project setup? I did not find anything on google on how to run the 1-click deployment from the commandline.

heitorlessa commented 4 years ago

I see.... we have that automated but it's essentially what made 1-click deployment possible, since the command line will prompt you for answers as opposed to what we got.

Perhaps you can expand on "many repetitions of performance measurements", but without knowing much allow me to explain what happens during our automated deployment so you can better understand Amplify POV:

1) 1-Click deployment configures an Amplify Console App (e.g. repository and branch as a source, etc.) 2) This will configure a Source (GitHub repo+branch), and detect we have amplify.yml at the root of our repository to automate our deployment (build/deploy steps for front-end and back-end) 2) During deployment, it'll initialize a new environment if there isn't one, then deploy the GraphQL API and Auth infra managed by Amplify (amplify folder config) 3) If successful, we export all deployed infra to System Manager Parameter Store 4) We use Serverless Application Model (SAM) framework to deploy our other back-ends, and look up Parameter Store to connect them (e.g. Loyalty needs to add a GraphQL resolver to an existing API) 5) Lastly, we build and deploy the front-end which grabs API and Auth config and bake into the Vue runtime

It's a fairly elaborate process as I wanted to show a more advanced deployment, but happy to expand further if that's the info you're after

heitorlessa commented 4 years ago

hey @SimonEismann - Just checking to see if that answers your question, or if you need me to expand on any specific topic.

SimonEismann commented 4 years ago

Still working on it, but so far I am positive that I should be able to get it to work. Big thanks already, your description helped demystify amplify/your build process quite a bit. I'll post a detailed update once I have it up and running.

SimonEismann commented 4 years ago

@heitorlessa - To elaborate a bit on what we are trying to do, we are currently working on a case study on performance regression testing of serverless applications where we are using this as the application under test. In order to get statistically significant results, we need to repeat experiments multiple times and in order to ensure that there is no cross-contamination between experiment runs, we need to tear down and re-deploy the application after each experiment run. Finally, in order to make the study reproducible and to avoid human error, we want to fully automate this deployment/teardown process.

Based on your explanation, it seems to me there are two ways to automate the deployment, 1) Automate the creation/deletion of an Amplify Console App or 2) Run all commands from the amplify.yml locally.

Solution 1 would a lot simpler, but unfortunately it seems that there is currently no API/SDK that would allow this (to be fair, it is probably unusual to automate the creation of Amplify Console Apps, as CI/CD is managed within the amplify console).

This lead me to solution 2. Here, I recreated the required dependencies in a docker container (I saw too late that there is an amplify build image, which probably does exactly this). Then the deployment consists of the following steps:

run amplifyPush.sh
export all variables as environment variables and to the system manager parameter store (analog to the amplify.yml)
build the frontend locally analog to the amplify.yml

configure the amplify project to skip the build step and to use vue + correct folder for frontend


IFS='|'
REACTCONFIG="{\
\"SourceDir\":\"src/frontend\",\
\"DistributionDir\":\"src/frontend/dist\",\
\"BuildCommand\":\"ls\",\
\"StartCommand\":\"npm run serve\"\
}"
AWSCLOUDFORMATIONCONFIG="{\
\"configLevel\":\"project\",\
\"useProfile\":false,\
\"profileName\":\"default\",\
\"accessKeyId\":\"${AWS_ACCESS_KEY_ID}\",\
\"secretAccessKey\":\"${$AWS_SECRET_ACCESS_KEY}\",\
\"region\":\"eu-central-1\"\
}"
AMPLIFY="{\
\"projectName\":\"awsserverlessairline\",\
\"defaultEditor\":\"None\"\
}"
FRONTEND="{\
\"frontend\":\"javascript\",\
\"framework\":\"vue\",\
\"config\":$REACTCONFIG\
}"
PROVIDERS="{\
\"awscloudformation\":$AWSCLOUDFORMATIONCONFIG\
}"

amplify configure project \ --amplify $AMPLIFY \ --frontend $FRONTEND \ --providers $PROVIDERS \ --yes


5. add S3 HTTP hosting to the amplify project (this was pretty annoying as there is no headless version of amplify add hosting :/)
6. run `amplify publish`

This results in a URL where I can access a functioning frontend that seems to communicate correctly with the backend (I am able to register new users and the mail with the confirmation code is send out). What's a little weird is that the frontend does not show up in the AWS GUI, there it still wants me to connect a frontend app. But since my use case does not require CI/CD, this should not be an issue, right?

heitorlessa commented 4 years ago

Aha! That's much clearer now - I do have one question though before I answer this.

This results in a URL where I can access a functioning frontend that seems to communicate correctly with the backend

Do you need all functionalities or one back-end like flight catalog suffices?

Flight catalog is deployed along with the front-end (amplifyPush), but loyalty, payment, integrations are deployed as part of the amplify.yml --- So if a single back-end suffices for that then there's an easier path for that

SimonEismann commented 4 years ago

The more functionality we have, the better. For example, it would also be interesting to investigate how the call to the external service (Stripe) influences the detection accuracy.

Tearing the application down via

make delete
amplify delete

seems to work fine, but I get the following error:

Error in deleting env:twitch
Stack with id amplify-awsserverlessairline-twitch-160135 does not exist
Stack with id amplify-awsserverlessairline-twitch-160135 does not exist
ValidationError: Stack with id amplify-awsserverlessairline-twitch-160135 does not exist
    at Request.extractError (/usr/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/protocol/query.js:50:29)
    at Request.callListeners (/usr/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
    at Request.emit (/usr/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
    at Request.emit (/usr/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/request.js:683:14)
    at Request.transition (/usr/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/usr/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /usr/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (/usr/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/request.js:38:9)
    at Request.<anonymous> (/usr/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/request.js:685:12)
    at Request.callListeners (/usr/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/sequential_executor.js:116:18)

It still correctly removes all CFN stacks , though. The variable $STACK_NAME does not contain twitch as I am using the develop branch at the moment. Is this due to my weird way of setting things up or something that is still hardcoded somewhere?

heitorlessa commented 4 years ago

Quick notes:

re: Stripe - You might be throttled, and worth knowing that Stripe can have responses as high as ~2s in the p99
- Possibly due to fraud and other detections for new transactions
As for the deployment, your $STACK_NAME is fine otherwise make delete would've failed
Within your script, do you initialize/create a new environment for every run?
What environments do you have available locally? Does that reflect what you have in your account? It could be out of sync
- Environments available and which one is being used for all operations: amplify env list
- All env details including Stack Name: amplify env list --details --json

As regards to the Amplify error, that stack name is wrong and reflects the stack associated with this project in my temp account - My guess is that you're either not initializing a new environment, or the CLI is picking up the wrong environment and you might need to explicitly switch environments (amplify env checkout).

Perhaps it's not using a clean environment on the iteration and instead tries to pick the original env this project was setup (twitch)

If that helps, feel free to DM me on Twitter so we can schedule a call to go over all topics you need help with for the research.

SimonEismann commented 4 years ago

Removing the existing stack solved the shutdown issue. For anything further I'll DM you and close the issue, thanks a lot for your help already!

aws-samples / aws-serverless-airline-booking

Deploying frontend via Amplify CLI #103