aws-amplify / amplify-cli

The AWS Amplify CLI is a toolchain for simplifying serverless web and mobile development.
Apache License 2.0
2.81k stars 821 forks source link

Custom AppSync Datasources #2722

Closed bduff9 closed 3 years ago

bduff9 commented 4 years ago

Is your feature request related to a problem? Please describe. I apologize if this already exists (and would be thrilled if it does) but could not find an answer after looking in the docs and past issues. We have a large production application that uses AppSync for all data-related needs. Most of this comes directly from an Aurora instance, however a growing amount come from other APIs such as lambda functions. To add one of these is not a huge deal, as we can add it to the JSON file for the datasources/functions/resolvers, however the issue is that these values are all unique per environment. Currently, this poses 2 challenges. First, when adding a datasource and merging to a main branch (i.e. develop to qa), we have to merge the new code into the environment, and then manually change the datasource info, commit this and then push it up using amplify push. Not a big deal but being a manual step, would be easy to miss. Second, we have our datasources for lambdas defined, and then in each resolver there is a path component that points to the current environment that also needs to be manually changed (i.e. resourcePath: '/dev/rest/of/path', to resourcePath: '/qa/rest/of/path',). This is also a manual step like the first part but is a bigger deal since future changes to this file could cause overwrites/conflicts.

Describe the solution you'd like Unsure if there is a current way to manage this, but some way to add the right datasources per environment, and then not lock the resourcePaths to the current environment to make deploys much easier/less prone to breaking.

Describe alternatives you've considered As mentioned, this has been working for a month or two, but is too manual and can cause hard to track issues (like it just did earlier today).

Additional context We might not be following the best practices here, so if there is a way to make this better/easier, I am open to that as well. Just could not find the answer after looking so hoping one of the experts here can point us in the right direction.

yuth commented 4 years ago

Most of this comes directly from an Aurora instance, however a growing amount come from other APIs such as lambda functions.

Are these functions provisioned using Amplify CLI? If these are provisioned using Amplify CLI, we expose an environment variable called ENV

To add one of these is not a huge deal, as we can add it to the JSON file for the datasources/functions/resolvers, however the issue is that these values are all unique per environment.

Where do add the JSON file? Is it in the lambda function itself or are you switching these using a custom stack?

bduff9 commented 4 years ago

No they are not provisioned with the CLI, they are a part of a seperate application, that we call out to from AppSync.

The JSON file is the appsync file that maps all the resolvers and functions and pipelines and datasources. So we add each of these as a datasource at the top below the existing Aurora datasource.

yuth commented 4 years ago

The JSON file is the appsync file that maps all the resolvers and functions and pipelines and datasources. So we add each of these as a datasource at the top below the existing Aurora datasource.

Is this inside the cloudformation file or is it something custom you have in your resolver?

bduff9 commented 4 years ago

I'm not sure I'm saying the correct terms here, but I'm only speaking about built-in stuff with amplify's API. What I am referencing is the file Amplify built for us in amplify/backend/api/API_NAME/stacks. Ours happens to be called API_NAME-PROJECT_NAME-rds.json. This file lists out all the datasources, functions, and resolvers, and which things depend on which others, and which VTL files to use for each resolver/function.

Within this file, since we list out all datasources, we have the first issue I mentioned which is the datasources change per environment.

Within the actual VTL files for resolvers, we have the second issue, which is the path of the lambda containing the specific environment name.

I hope this clarifies but if not, let me know how I can better describe the setup that amplify gives you when you run amplify add api and then choose to integrate with an aurora serverless instance.

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

bduff9 commented 4 years ago

Any more thoughts on this or info I can provide? Still have these described issues from my side, but could be lack of knowledge on my part.

kaustavghosh06 commented 3 years ago

@bduff9 You can maybe use a custom resolver with a custom datasource for your use-case. Have you gotten a chance to look at - https://docs.amplify.aws/cli/graphql-transformer/resolvers Note: You'd have to write CloudFormation definitions for your custom data-source.

stale[bot] commented 3 years ago

This issue has been automatically closed because of inactivity. Please open a new issue if you are still encountering problems.

github-actions[bot] commented 3 years ago

This issue has been automatically locked since there hasn't been any recent activity after it was closed. Please open a new issue for related bugs.

Looking for a help forum? We recommend joining the Amplify Community Discord server *-help channels for those types of questions.