RFC - Local development and testing

[yuth]

Amplify CLI with the support of GraphQL transformer, makes it simple to develop a GraphQL API. To see this API in action, Amplify CLI requires customers to push to the cloud. Pushing changes to cloud is time consuming and makes iterative development & debugging less than ideal.

We are looking to support running AppSync APIs locally and make the iterative development-test-push processes faster. This RFC is a proposal for feedback on workflow and requirements as part of this effort. Please respond with any feedback on the current proposals or missing scenarios/ideas.

We propose adding new functionality to Amplify CLI, which is intended to provide the following functionality:

• Allow customer to run AppSync test server locally
• Detect changes to GraphQL schema and resolvers locally and hot-reload the server
• Run codegen to generate statements and types from the local server
• Generate configuration (for JavaScript, iOS, and Android platforms) files that include the local endpoint for AppSync service
• Host GraphiQL UI on the test server to play with API

Example use case

The below example use case shows how local testing can be used. Let’s imagine a you have an Amplify Project initialized with React as front-end. You add a new API called MyGraphQLAPI:

$ amplify add api
? Please select from one of the below mentioned services (Use arrow keys)
❯ GraphQL
? Please select from one of the below mentioned services GraphQL
? Provide API name: MyGraphQLAPI
? Choose an authorization type for the API: API key
? Do you have an annotated GraphQL schema? No
? Do you want a guided schema creation? Yes
? What best describes your project: Single object with fields (e.g., “Todo”  with ID, name, description)
? Do you want to edit the schema now? Yes
Please edit the file in your editor: ./amplify/backend/api/rds/schema.graphql
? Press enter to continue

GraphQL schema compiled successfully.
Edit your schema at ./amplify/backend/api/MyGraphQLAPI/schema.graphql
or place .graphql files in a directory at ./amplify/backend/api/MyGraphQLAPI/schema.graphql
Successfully added resource MyGraphQLAPI locally

Some next steps:
"amplify test api " (or some similar command that might be implicit in the push flow) will run AppSync test server locally on your machine and allow you to
play with the GraphQL API
"amplify push" will build all your local backend resources and provision it
in the cloud
"amplify publish" will build all your local backend and frontend resources (if
you have hosting category added) and provision it in the cloud

Once the API is added, you can start the AppSync test server by running the following command. When there are resources other than GraphQL API in your project, those will be pushed to cloud first followed by starting AppSync test server.

$ amplify test api
Current Environment: dev

| Category | Resource name | Operation | Provider plugin   |
| -------- | ------------- | --------- | ----------------- |
| Api      | MyGraphQLAPI  | Create    | awscloudformation |
| Auth     | cognito70e6   | Create    | awscloudformation |

The following resources can not be tested locally

| Category | Resource name | Operation | Provider plugin   |
| -------- | ------------- | --------- | ----------------- |
| Auth     | cognito70e6   | Create    | awscloudformation |

Do you want to push these resources to the cloud: Yes

UPDATE_IN_PROGRESS my-cool-project-dev-12222 AWS::CloudFormation::Stack Wed May 01 2019 11:40:26 GMT-0700 (Pacific Daylight Time) User Initiated
⠸ Updating resources in the cloud. This may take a few minutes...

CREATE_IN_PROGRESS cognito70e6 AWS::CloudFormation::Stack Wed May 01 2019 11:40:34 GMT-0700 (Pacific Daylight Time) Resource creation Initiated
CREATE_IN_PROGRESS cognito70e6 AWS::CloudFormation::Stack Wed May 01 2019 11:40:32 GMT-0700 (Pacific Daylight Time)
⠴ Updating resources in the cloud. This may take a few minutes...
...
...
...
...
⠦ Updating resources in the cloud. This may take a few minutes...
✔ All resources are updated in the cloud

? Do you want to generate code for your newly created GraphQL API: Yes
? Choose the code generation language target: javascript
? Enter the file name pattern of graphql queries, mutations and subscriptions: src/graphql/**/*.js
? Do you want to generate/update all possible GraphQL operations - queries, mutations and subscriptions: Yes
? Enter maximum statement depth [increase from default if your schema is deeply nested]: 2

Starting AppSync test server for MyGraphQLAPI
✔ GraphQL schema compiled successfully.
Edit your schema at ./amplify/backend/api/MyGraphQLAPI/schema.graphql
or place .graphql files in a directory at ./amplify/backend/api/MyGraphQLAPI/schema.graphql
Generating statements
Generating  src/aws_exports.js
You can test your AppSync API by opening http://localhost:8090/

The AppSync test server will watch the schema and resolvers files for change and reload when it detects changes in any of these files. If there are errors in the schema or resolvers, those will be shown in the terminal

Change detected in ./amplify/backend/api/MyGraphQLAPI/schema.graphql
Rebuilding the schema using GraphQL transformer
✖ GraphQL schema failed to compile.
Edit your schema at ./amplify/backend/api/MyGraphQLAPI/schema.graphql or
place .graphql files in a directory at ./amplify/backend/api/MyGraphQLAPI/schema/

If the change did not have any error then then the server will reload with updated schema and resolvers

Change detected in ./amplify/backend/api/MyGraphQLAPI/schema.graphql
Rebuilding the schema using GraphQL transformer
✔GraphQL schema compiled successfully.
✔ Reloaded successfully.

Auth support

The AppSync testing server will support API Key and Cognito user pools. Each request will have to either have x-api-key for API Key and authorization with JWT Token for Cognito User Pools. The JWT token has to be obtained from Cognito by either using Amplify.Auth.currentSession().getAccessToken().getJwtToken() in Javascript or using AWSMobileClient.getInstance().getTokens().getAccessToken().getTokenString() in Android.

VTL Error handling

AppSync local testing server watches the files the local file system and reloads the server when watched files changes. When the server reloads, it will parse all the VTL template and it will print an error in the console if there are an error in VTL template.

Change detected in ./amplify/backend/api/MyGraphQLAPI/resolvers/Mutation.addTodo.req.vtl
Error: Error parsing file amplif/backend/api/MyCoolAPI/resolvers/Mutation.addTodo.req.vtl
Lexical error on line 2. Unrecognized text.
...pochMilliSeconds())#set( $values = $ctx
----------------------^
✖ Reload failed

After fixing the VTL template error and saving the changes, AppSync test server will reload the template and if there are no further erros, the server will reload successfully.

Change detected in ./amplify/backend/api/MyGraphQLAPI/resolvers/Mutation.addTodo.req.vtl
✔ Reloaded successfully.

We plan to release a companion VSCode extension that can parse and show errors in the your VTL templates in the editor.

Please comment with any additional thoughts not covered in the above list. Thank you.

[mwarger]

This may be somewhat related, but specifically regarding any updates to the type generation. I would like to throw out an idea for updating the code generation to use apollo cli, if possible. This provides watch mode for type generation and also automatically extracts types for sub-queries. It also supports having graphql operations co-located with components (using gql tag). I mentioned it here regarding a current bug - https://github.com/aws-amplify/amplify-cli/issues/1325#issuecomment-485846397

[undefobj]

Do you mean GrraphQL Code Generator? https://graphql-code-generator.com If so we are in communication with that team already. @mwarger

[mwarger]

I did not, actually meant https://github.com/apollographql/apollo-tooling#apollo-clientcodegen-output

But, what you linked seems perfectly great and a good upgrade from what currently exists. I will give that a look and follow-up later.

Edit: I remember this now, @undefobj - I did look at this previously and thought how cool it would be if there was an amplify plugin for this. Good stuff 👍

[chrisco512]

Will data persist between runs? Does this have a local DynamoDB instance running in order to achieve this? Currently I rely heavily on CloudWatch logs for debugging. Will this sort of logging be available locally?

[davekiss]

How would this work with @function resolvers or pipeline resolvers? Will it also pipe requests to the local lambda functions that amplify is aware of?

[undefobj]

@chrisco255

Will data persist between runs?

Do you need this to be supported? If so how much data locally?

Does this have a local DynamoDB instance running in order to achieve this?

Right now it doesn't have anything, we are just getting feedback and requirements on the DX that customers are looking for. We haven't decided on an implementation just yet (though we do have ideas).

Currently I rely heavily on CloudWatch logs for debugging. Will this sort of logging be available locally?

What would be your preferred experience here? Logging to a file, stdout, or something else?

[undefobj]

How would this work with @function resolvers or pipeline resolvers? Will it also pipe requests to the local lambda functions that amplify is aware of?

@davekiss We didn't put the Function category into the RFC but we are thinking about it, and do have some rudimentary testing of Lambda already in the CLI: https://aws-amplify.github.io/docs/js/react#testing-serverless-functions

When the GraphQL Transformer has Pipeline resolver support in the future we'll look at supporting that locally but there is still more design work that is needed in that area. That being said we want to be able to support custom resolvers written in VTL which is supported today and you could write Pipeline resolvers there. That being the case what would be a desirable experience for you when using this?

[chrisco512]

@undefobj Ideally, I would like data to persist between runs, with the ability to flush as needed. At a minimum, I would like an easy format for hydrating data on start.

As far as logging goes, stdout is decent and the option to pipe to a file would be great, but I also like CloudWatch's model of being able to expand each step of the query from a web page, but I realize that would require additional UI work.

[sthulb]

I’ve had a lot of customers ask about local unit testing of their VTL. They would like to have confidence in the VTL code they write.

[buggy]

Is it possible to implement this in a way that's compatible with SAM CLI? SAM CLI already has support for running the API Gateway locally. This could benefit both SAM CLI users (getting access to AppSync locally) and Amplify CLI users (being able to use the API Gateway from SAM).

[wtrocki]

Allow customer to run AppSync test server locally

Does this mean running some docker container on local machine and connecting to AWS dynamodb or it will be fully local experience with some in mem database?

It will be cool to be able to still connect with DB or other services etc. running on AWS

[yuth]

@chrisco255,

Ideally, I would like data to persist between runs, with the ability to flush as needed. At a minimum, I would like an easy format for hydrating data on start.

We are thinking of using Dynamo DB local and data can be persisted between the runs.

I’ve had a lot of customers ask about local unit testing of their VTL. They would like to have confidence in the VTL code they write.

@sthulb we could add support for snapshot testing the VTL

Is it possible to implement this in a way that's compatible with SAM CLI? SAM CLI already has support for running the API Gateway locally. This could benefit both SAM CLI users (getting access to AppSync locally) and Amplify CLI users (being able to use the API Gateway from SAM).

@buggy Our plan is to to write the AppSync test server such that it can also be invoked using an API to allow other CLI/Framework can integrate AppSync testing

Does this mean running some docker container on local machine and connecting to AWS dynamodb or it will be fully local experience with some in mem database?

@wtrocki Our plan is to write this using NodeJS and use DynamoDB.

It will be cool to be able to still connect with DB or other services etc. running on AWS

We plan expose an API in the test server to add additional data sources. This would let users add additional data sources or change the implementation of existing data sources.

[sthulb]

@yuth SAM CLI currently doesn’t have any VTL support. The APIGW implementation at the moment is nothing more than a basic webserver.

I’m happy to talk to the team about the requirements: thulsimo@ on chime/email

[yareyaredesuyo]

In serverless, similar things can be done using this kind of plugin.

https://github.com/aheissenberger/serverless-appsync-offline

[0xR]

It's done! https://aws.amazon.com/blogs/aws/new-local-mocking-and-testing-with-the-amplify-cli/

[oste]

Any plans to support RDS local mocking? Actually considering a migration to Dynamo just for this :)

[kaustavghosh06]

@oste We don't have immediate plans for supporting local RDS mocking yet, but I'll add it to our backlog and look for for customer interest/feedback out here before we execute on it.

[oste]

@kaustavghosh06 thanks for the heads up. Maybe a good stopgap would be to have the ability to update/push a single resolver. Gotta think that could make pushing a resolver change go from 2-3 minutes to 2-3 seconds.

[mdepascale]

@kaustavghosh06 are there any update for supporting local RDS mocking?

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[askdesigners]

I'd love to see a way to trigger lambda resolvers with this approach as well. :)

[Bulletninja]

Not exactly done. If you create a custom resolver, say for a mutation, and from within that lambda you then try and call appsync or dynamodb directly it hangs (it is never able to connect to the DB). See https://github.com/aws-amplify/amplify-cli/issues/4072