Is it possible to use the same AppSync api across multiple serverless projects?

[EwanValentine]

I have several microservices, all of which exposed via AppSync. At the moment, I have one AppSync project, and when I deploy a service, it extracts the graphql config, uploads it to S3. Then the AppSync project pulls down the files and rebuilds. But it's a bit convoluted, and difficult to run locally. So I wondered if it was possible to 're-use' the same AppSync api, perhaps by giving it the same api name in the config?

[EwanValentine]

@sid88in any thoughts on this?

[sid88in]

not sure. did you try this out?

[EwanValentine]

@sid88in we did! Unfortunately it created another AppSync API with a different unique key.

I wonder if it would be possible, internally to provide a 'key', or maybe an ARN. If an AppSync key or Arn is provided in the custom config. It adds the config to the existing AppSync API instead?

[EwanValentine]

Willing to help out if you think that this is doable or possible

[bboure]

@EwanValentine I'd like to make sure I understood your issue.

So, you have several microservices all exposed through the same graphql. Are all microservices in different stacks/serverless.yml? If so, what you want it to deploy all microservices individually when updated and update only the part of graphql that corresponds to it? (probably a query or mutation field)

[EwanValentine]

@bboure they're in different stacks/serverless.yml files, yes. And ideally yes, that might be a crazy idea. We're doing it manually at the moment. So when we deploy a 'graphql service', we upload a GraphQL schema, a yaml config file with data sources, and any vtl templates into an S3 bucket. Then that kicks off a Codebuild pipelines to merge them all into one AppSync project.

It's a bit convoluted, and means it's tricky to run things in isolation. So I was doing some research to see if there was a better way, basically. Hope this helps!

[bboure]

@EwanValentine I assume that you are using lambdas for your microservices?

What about this:

Have your microservices only deploy a lambda with your code. No Appsync involved here. In your AppSync stack, your datasources call a proxy lambda whose only job is to call each individual microservice lambda with the received payload and return its result.

That way, each stack is independent. Your proxy lambda will likely never change and remain as it is (unless you need to add a new microservice). You just need to focus on deploying your microservices lambdas.

Of course this has a couple of drawbacks since each microservices will involve calling 2 lambdas each time: This will add a bit of latency and, of course, cost. To reduce that, your proxy lambda could be just one same lambda that receives as input the microservice lambda name that has to be called. Doing that should at least reuse it and keep it warm.

Example:

• your microservice stack "sendEmail" has a lambda called sendEmailLambda with your code.
• your appsync stack has a mutation "sendEmail" with a datasource that calls a proxyLambda.
• your sendEmail template adds the microservice name to the payload payload: {"microservice: "sendEmailLambda", microservicePayload: $utils.toJson($context)}
• your lambda proxyLambda calls the lambda received in the payload: microservice and returns its result directly.

Once everything is in place, when your sendEmail service changes, you just need to update the sendEmailLambda function. Your appsync stack will not change.

Would that be suitable for you?

[kkesley]

I'm currently having this problem as well.

My project looks like this:

/api
   /some-api-1
      api-1-function-1 (every function is a lambda)
      api-1-function-2
      serverless.yml
   /some-api-2
      api-2-function-1
      api-1-function-2
      serverless.yml
serverless.yml

I used that for API Gateway and now, I want to change it to appsync.

My problem is that I used to make an API Gateway Resource in the root serverless.yml and refer it in the child serverless.yml. Note: I need to deploy root serverless.yml first, then the children

it'd be nice to reuse the graphql api in the child serverless.yml

My main goal was to keep the structure flexible so it'd be easy to add new api / edit existing apis without building all of my code (if I change some-api-1, i don't want to rebuild some-api-2)

[bboure]

@kkesley @EwanValentine

I had a second look at this, I think that what you need is to be able to manually override this parameter, and specify an AppSync arn from an external stack In addition to that, you need to be able to define the dataSources in a child stack, without deploying a new AppSync API.

Now, that sounds like something possible, however, there is much more involved here. For example, the GraphQl schema. Ideally, the schema part for your child stacks should be defined there too, with the data source. But that means that whenever you want to deploy a child stack, you will need to pull the schema form the parent and siblings' stacks and merge them, in order to push the full schema (schema in AppSync is kinda "atomic"). This does not sound easy and prone to issues: deploying a microservice could affect others. I am not sure how we could do that easily.

Any suggestions?

[kkesley]

@bboure regarding the schema, would it be much simpler if we just manage it in the parent stack? in my opinion, this is appsync's limitation rather than this library's.

[bboure]

@kkesley I also see it as an AppSync limitation rather than the plugin.

If you are going to manage the schema in the parent stack, then what I can suggest you instead and would currently be supported is the following:

• Have all the AppySinc config, the schema and all the data sources in your parent stack.
• Have your lambda functions in the "child" stacks.
• In parent stack, reference your Lambda functions from the child stacks with their ARN.

To make this easy, you can take advantage of the Output feature from Cloudformation/Serverless.

This way, you only need to touch your AppSync stack if the structure our your api (schema) changes. You microservices will then only take care of deploying the lambdas when your code logic changes.

How does that sound?

[kkesley]

@sid88in that's one way to do it. However, it will clutter the parent stack with data sources and mapping templates..

Furthermore, parent stack would be dependent on child stacks. which means, if you want to deploy it again in a new environment, you would deploy child stacks first and then the parent stack. In my opinion, it's less intuitive.

[EwanValentine]

Sorry for the late response! Just read back through all of this. I agree it's an AppSync limitation, probably well out of scope for this particular project. Thanks for looking into it though, I'll take on board some of the other suggestions above

[EwanValentine]

Feel free to close, unless you think there's value in keeping this discussion on-going

[andrejunges]

any updates on this? facing this issue currently.. seems like we still don't have a "perfect"/recommended solution.

[bboure]

@andrejunges it really depends on your use case. can you elaborate?

[andrejunges]

sure, currently we have multiple services exposing it's own Appsync API... But we'd like to have them all together, merge these individual schemas into one centralized appsync api.

it seems apiId (which was deprecated) could help doing that, but based on the replies here it wouldn't merge the schemas defined in each service.

Not sure what's the best solution for now, but our ideal solution would be to define the schema, mapping templates and datasources in each service, but deploy them all together using the same AppSync API.

[bboure]

@andrejunges What you probably want is Schema federation This is currently not supported by AppSync, but there is a request for it: https://github.com/aws/aws-appsync-community/issues/35

However someone suggested using this: https://github.com/0xR/graphql-transform-federation

I have not tried it myself but that might be something you can explore?

Since this seems like something of interest, I will re-open this issue.

[andrejunges]

@bboure thanks again for the reply. I saw that on my researches and I agree that would be the ideal solution - though not sure how well it'd work and was trying to get some feedback first.

when I read about the apiId it seemed like a better solution until appsync implements something like apollo federation or at least provide support for it - https://github.com/serverless-components/aws-app-sync#create-or-reuse-apis. Is it accurate apiId do not merge the schemas? or what was the reason this library deprecated it?

[SoimanAndrei]

Any news about supporting https://github.com/serverless-components/aws-app-sync#create-or-reuse-apis ?

[bboure]

@SoimanAndrei Please see this thread: https://github.com/sid88in/serverless-appsync-plugin/issues/300 I gave some clues on how this might be achieved. I have not tried it though.

[tomaszdudek7]

@kkesley I just hit the same issue with a very similar structure of directories you've mentioned in this comment.

Have you somehow fixed / found a workaround on this?

For now I'm thinking of:

• using Serverless with JS/TS instead of YAML (if the serverless-appsync-plugin supports it...) to allow me to package and reuse templates as well as use the actual programming language to tinkle with sls templates in a sane way
• changing the deployment order (going from apigateway-ish style of 1. top level serverless.yml with resources -> 2. service abc to 1. top level serverless.js with resources -> 2. service abc -> 3. serverless.js with appsync)
• merging all the schema.graphql from my "microservices" into one using this tool
• merging the mappingTemplates from my "microservices" into one using ...not sure what yet, but the expectation is that I dynamically create something as follows (and then convert the yaml into js):

  mappingTemplates:
  #### this comes from service abc
  - type: Query
  field: exampleQueryField
  dataSource: exampleDataSource
  request: @import_vtl_file_from_package_abc
  response: @import_vtl_file_from_package_abc
  #### this comes from service def
  - type: Query
  field: exampleQueryFieldTwo
  dataSource: exampleDataSourceTwo
  request: @import_vtl_file_from_package_def
  response: @import_vtl_file_from_package_def