(appsync): Cross-stack resources -> Fn::GetAtt references undefined

ruggero-balteri commented 3 years ago

I have 2 stacks:

API-stack which contains the Appsync API
SQS-stack, which contains a SQS resource (other stack are removed for simplicity)

I want to simply pass the SQS queue (defined in the SQS stack) to the API stack

The code synthesizes correctly, but it fails at deployment with error:

 ❌  AaaApiStack failed: Error [ValidationError]: Template error: instance of Fn::GetAtt references undefined resource StudentQueue68D52AA1
    at Request.extractError (/usr/local/lib/node_modules/aws-cdk/node_modules/aws-sdk/lib/protocol/query.js:50:29)
    at Request.callListeners (/usr/local/lib/node_modules/aws-cdk/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
    at Request.emit (/usr/local/lib/node_modules/aws-cdk/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
    at Request.emit (/usr/local/lib/node_modules/aws-cdk/node_modules/aws-sdk/lib/request.js:688:14)
    at Request.transition (/usr/local/lib/node_modules/aws-cdk/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/usr/local/lib/node_modules/aws-cdk/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /usr/local/lib/node_modules/aws-cdk/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (/usr/local/lib/node_modules/aws-cdk/node_modules/aws-sdk/lib/request.js:38:9)
    at Request.<anonymous> (/usr/local/lib/node_modules/aws-cdk/node_modules/aws-sdk/lib/request.js:690:12)
    at Request.callListeners (/usr/local/lib/node_modules/aws-cdk/node_modules/aws-sdk/lib/sequential_executor.js:116:18) {
  code: 'ValidationError',
  time: 2021-01-31T13:22:49.329Z,
  requestId: '7f7feb96-4b35-46b1-bc67-e3c3f0d1aa2f',
  statusCode: 400,
  retryable: false,
  retryDelay: 445.1091591510583
}
Template error: instance of Fn::GetAtt references undefined resource StudentQueue68D52AA1

Reproduction Steps

cdk.ts

#!/usr/bin/env node
import "source-map-support/register";
import * as cdk from "@aws-cdk/core";
import { SQSStack } from "../lib/sqs/stack";
import { ApiStack } from "../lib/api/stack";
const app = new cdk.App();

const env = {
  region: "us-east-1",
  account: "446416672052",
};
const SQSStack = new SQSStack(
  app,
  "AaaSQSStack",
  {
    env,
  }
);

const apiStack = new ApiStack(app, "AaaApiStack", {
  env,
  SQSSqs: SQSStack.studentQueue,
});

API-stack

import * as cdk from "@aws-cdk/core";
import * as node from "@aws-cdk/aws-lambda-nodejs";
import * as appsync from "@aws-cdk/aws-appsync";

import * as sqs from "@aws-cdk/aws-sqs";
import { ResolvableField } from "@aws-cdk/aws-appsync";
import * as fs from "fs";
import { join } from "path";

export interface ApiStackProps extends cdk.StackProps {
  SQSSqs: sqs.IQueue;
}

export class ApiStack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string, props: ApiStackProps) {
    super(scope, id, props);

    const account = props.env?.account || "";
    const region = props.env?.region || "";

    const api = new appsync.GraphqlApi(this, "AaaApi", {
      name: "AaaApi",
      xrayEnabled: true,
    });

    const mockQuery = new appsync.ResolvableField({
      returnType: appsync.GraphqlType.string(),
      dataSource: api.addNoneDataSource("none"),
      requestMappingTemplate: appsync.MappingTemplate.dynamoDbScanTable(),
      responseMappingTemplate: appsync.MappingTemplate.dynamoDbResultList(),
    });
    api.addQuery("mockQuery", mockQuery);

    // Matching Calling Recording Stack

    const { SQSSqs } = props;

    const sqsDs = api.addHttpDataSource(
      "SqsDs",
      `https://sqs.${region}.amazonaws.com/`
    );

    const requestMappingTemplate = fs
      .readFileSync(
        join(
          __dirname,
          "/resolvers/sqs/sendSqs/request.vtl"
        ),
        {
          encoding: "utf-8",
        }
      )
      .replace("ACCOUNT_ID", account)
      .replace("QUEUE_NAME", SQSSqs.queueName);

    const sendSqs = new appsync.ResolvableField({
      returnType: appsync.GraphqlType.string(),
      dataSource: sqsDs,
      requestMappingTemplate: appsync.MappingTemplate.fromString(
        requestMappingTemplate
      ),
      responseMappingTemplate: appsync.MappingTemplate.fromFile(
        join(
          __dirname,
          "/resolvers/sqs/sendSqs/response.vtl"
        )
      ),
    });

    api.addMutation("sendSqsStudentRequest", sendSqs);
    // const role = new iam.Role(stack, 'Role', {
    //   assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'),
    // });
    // api.grant(role, appsync.IamResource.custom('types/Mutation/fields/updateExample'), 'appsync:GraphQL')
  }
}

SQS stack

import * as cdk from "@aws-cdk/core";
import * as node from "@aws-cdk/aws-lambda-nodejs";
import * as appsync from "@aws-cdk/aws-appsync";
import * as sqs from "@aws-cdk/aws-sqs";
import { ResolvableField } from "@aws-cdk/aws-appsync";
import * as fs from "fs";
import { join } from "path";

export class SQSStack extends cdk.Stack {
  readonly studentQueue: sqs.IQueue;

  constructor(scope: cdk.Construct, id: string, props: cdk.StackProps) {
    super(scope, id, props);

    this.studentQueue = new sqs.Queue(this, "StudentQueue");
  }
}

What did you expect to happen?

CDK should have handled the routing and make sure that the appropriate resources (e.g. StudentQueue68D52AA1) was exported correctly

What actually happened?

The appropriate resource was not exported from SQS-stack

Environment

CDK CLI Version : 1.87.1
Framework Version:
Node.js Version: v14.15.4
OS : 10.15.7 (19H114)
Language (Version): TypeScript 3.9.7

Other

This issue might be similar https://github.com/aws/aws-cdk/issues/3619

Synth SQS stack

Resources:
  StudentQueue68D52AA1:
    Type: AWS::SQS::Queue
    Metadata:
      aws:cdk:path: AaaSQSStack/StudentQueue/Resource
  CDKMetadata:
    Type: AWS::CDK::Metadata
    Properties:
      Modules: aws-cdk=1.87.1,@aws-cdk/aws-appsync=1.87.1,@aws-cdk/aws-cloudwatch=1.87.1,@aws-cdk/aws-iam=1.87.1,@aws-cdk/aws-kms=1.87.1,@aws-cdk/aws-sqs=1.87.1,@aws-cdk/cloud-assembly-schema=1.87.1,@aws-cdk/core=1.87.1,@aws-cdk/cx-api=1.87.1,@aws-cdk/region-info=1.87.1,jsii-runtime=node.js/v14.15.4
    Metadata:
      aws:cdk:path: AaaSQSStack/CDKMetadata/Default

Synth API stack

Resources:
  AaaApi72C55F6E:
    Type: AWS::AppSync::GraphQLApi
    Properties:
      AuthenticationType: API_KEY
      Name: AaaApi
      XrayEnabled: true
    Metadata:
      aws:cdk:path: AaaApiStack/AaaApi/Resource
  AaaApiSchema7879F737:
    Type: AWS::AppSync::GraphQLSchema
    Properties:
      ApiId:
        Fn::GetAtt:
          - AaaApi72C55F6E
          - ApiId
      Definition: >
        schema {
          query: Query
          mutation: Mutation
        }

        type Query {
          mockQuery: String
        }

        type Mutation {
          sendSqsStudentRequest: String
        }
    Metadata:
      aws:cdk:path: AaaApiStack/AaaApi/Schema
  AaaApiDefaultApiKeyDF40679C:
    Type: AWS::AppSync::ApiKey
    Properties:
      ApiId:
        Fn::GetAtt:
          - AaaApi72C55F6E
          - ApiId
    DependsOn:
      - AaaApiSchema7879F737
    Metadata:
      aws:cdk:path: AaaApiStack/AaaApi/DefaultApiKey
  AaaApinone4B86ED1B:
    Type: AWS::AppSync::DataSource
    Properties:
      ApiId:
        Fn::GetAtt:
          - AaaApi72C55F6E
          - ApiId
      Name: none
      Type: NONE
    Metadata:
      aws:cdk:path: AaaApiStack/AaaApi/none/Resource
  AaaApiSqsDsServiceRole12FDDD19:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
          - Action: sts:AssumeRole
            Effect: Allow
            Principal:
              Service: appsync.amazonaws.com
        Version: "2012-10-17"
    Metadata:
      aws:cdk:path: AaaApiStack/AaaApi/SqsDs/ServiceRole/Resource
  AaaApiSqsDs580AB52D:
    Type: AWS::AppSync::DataSource
    Properties:
      ApiId:
        Fn::GetAtt:
          - AaaApi72C55F6E
          - ApiId
      Name: SqsDs
      Type: HTTP
      HttpConfig:
        Endpoint: https://sqs.us-east-1.amazonaws.com/
      ServiceRoleArn:
        Fn::GetAtt:
          - AaaApiSqsDsServiceRole12FDDD19
          - Arn
    Metadata:
      aws:cdk:path: AaaApiStack/AaaApi/SqsDs/Resource
  AaaApiQuerymockQueryResolver3471C8E8:
    Type: AWS::AppSync::Resolver
    Properties:
      ApiId:
        Fn::GetAtt:
          - AaaApi72C55F6E
          - ApiId
      FieldName: mockQuery
      TypeName: Query
      DataSourceName: none
      Kind: UNIT
      RequestMappingTemplate: '{"version" : "2017-02-28", "operation" : "Scan"}'
      ResponseMappingTemplate: $util.toJson($ctx.result.items)
    DependsOn:
      - AaaApinone4B86ED1B
      - AaaApiSchema7879F737
    Metadata:
      aws:cdk:path: AaaApiStack/AaaApi/QuerymockQueryResolver/Resource
  AaaApiMutationsendSqsStudentRequestResolver9F64A029:
    Type: AWS::AppSync::Resolver
    Properties:
      ApiId:
        Fn::GetAtt:
          - AaaApi72C55F6E
          - ApiId
      FieldName: sendSqsStudentRequest
      TypeName: Mutation
      DataSourceName: SqsDs
      Kind: UNIT
      RequestMappingTemplate:
        Fn::Join:
          - ""
          - - >-
              {
                "version": "2018-05-29",
                "method": "GET",
                "resourcePath": "/446416672052/
            - Fn::GetAtt:
                - StudentQueue68D52AA1
                - QueueName
            - >-
              ",
                "params": {
                  "query": {
                    "Action": "SendMessage",
                    "Version": "2012-11-05",
                    "MessageBody": "$util.urlEncode($util.toJson($ctx.args.input))"
                  }
                }
              }
      ResponseMappingTemplate: >-
        #set( $result = $utils.xml.toMap($ctx.result.body) )

        {
          "id": "$result.SendMessageResponse.SendMessageResult.MessageId",
          "email": "$ctx.args.input.email",
          "message": "$ctx.args.input.message",
        }
    DependsOn:
      - AaaApiSchema7879F737
      - AaaApiSqsDs580AB52D
    Metadata:
      aws:cdk:path: AaaApiStack/AaaApi/MutationsendSqsStudentRequestResolver/Resource
  CDKMetadata:
    Type: AWS::CDK::Metadata
    Properties:
      Modules: aws-cdk=1.87.1,@aws-cdk/aws-appsync=1.87.1,@aws-cdk/aws-cloudwatch=1.87.1,@aws-cdk/aws-iam=1.87.1,@aws-cdk/aws-kms=1.87.1,@aws-cdk/aws-sqs=1.87.1,@aws-cdk/cloud-assembly-schema=1.87.1,@aws-cdk/core=1.87.1,@aws-cdk/cx-api=1.87.1,@aws-cdk/region-info=1.87.1,jsii-runtime=node.js/v14.15.4
    Metadata:
      aws:cdk:path: AaaApiStack/CDKMetadata/Default

This is :bug: Bug Report

ruggero-balteri commented 3 years ago

found the problem. In API-stack import the property Queue from SQS-stack and I use it inside the .replace() function (not inside a cdk construct). Cdk, which should be in charge or creating the "wiring" between stacks, ignores the properties that are not used inside stacks, including the IQueue. As a workaround, I just instantiated the following resource (which I am not using) to setup the wiring.

    new cdk.CfnOutput(this, "WorkaroundForWiring", {
      value: SqsSqs.queueName,
    });

Since CfnOutput is a construct, the queueName is now exported from SQS stack and imported in API stack and can be use.

Why is this workaround necessary? Shouldn't cdk take care of this situations without the need of a component?

MrArnoldPalmer commented 3 years ago

@ruggero-balteri you're right it should. This is likely a bug in appsync.

BryanPan342 commented 3 years ago

@MrArnoldPalmer not quite sure why it would be an appsync bug tho? I'm also not that much of an expert on cross-stack implementation under the hood.

But what I'm guessing is happening is that CDK creates a token that resolves later, and in the case of this bug report, the token isn't being created and thus not resolved properly?

MrArnoldPalmer commented 3 years ago

I'm not sure if it is, but I was thinking there is a possibility we are doing something wrong referencing the IQueue that prevents it from being properly exported from the stack that makes it, which is required to reference it in the api stack.

adworacz commented 3 years ago

Just ran into this myself, in this case referencing a Lambda ARN from one stack in a custom resource (wraps CfnCustomResource) in another stack.

Luckily, that other stack only had a single construct in it, so I simply replicated the construct into the same stack that creates the Lambda.

Obviously this a corner case. The workaround posted above might be a solution in this case, but I haven't tried it.

BryanPan342 commented 3 years ago

@adworacz are you comfortable sharing how you were referencing the Lambda ARN? More examples will definitely help resolve this case!

adworacz commented 3 years ago

Yup it was something like the following:


class LambdaStack extends DeploymentStack {
   public readonly lambdaArn: string

   constructor(...) {
     const lambdaFunction = new Function(this, ...)
     this.lambdaArn = lambdaFunction.functionArn
     ...
   }
}

class CustomResourceStack extends DeploymentStack {...}

const lambdaStack = new LambdaStack(...)

const customResourceProps = { 
    uri: `arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/${lambdaStack.lambdaArn}/invocations` 
}
const customResourceStack = new CustomResourceStack(..., customResourceProps)

BryanPan342 commented 3 years ago

Okay I think this is an issue with the core library. And what seems to be the issue is whenever there is string interpolation using a token, CDK treats the token as undefined?

@MrArnoldPalmer I'm pretty sure this is the error, but would like your thoughts

MrArnoldPalmer commented 3 years ago

This could definitely be a problem with tokens and string interpolation though that is supposed to work in general. It would require more investigation on our end I think to rule out a problem with the AppSync constructs still.

github-actions[bot] commented 2 years ago

This issue has not received any attention in 1 year. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

antoniordz96 commented 1 year ago

Hey folks I just hit the same issue here. In our organization we are not allowed to generate our own KMS keys rather request via Service catalog. I used the L1 construct that is in a BaseStack to generated the SC product which outputs the Kms ARN. In that stack I create the Ikey using the fromArn method. Just deploying that stack alone works as expected.

In my second stack I try to pass the public readonly Ikeyfrom the base stack and hit the error:

Template error: instance of Fn::GetAtt references undefined resource DeltaKmsProvisionedProduct

I got around this by just using CFNOutputs. Would be nice if we could just reference the public Ikey from the class

ncnewton17 commented 5 months ago

Hi all, I also hit the same (or at least very similar) issue. I'm getting the error: Template error: instance of Fn::GetAtt references undefined resource EmrSecurityGroupFGACF683B51A on deploy. No error on stack build. I have basically the following situation:

# stack A
...
this.fgacSecurityGroup = new SecurityGroup(...)
...

# app.ts
const stackA = new StackA(...)

new StackB(... {
  fgacSecurityGroup: stackA.fgacSecurityGroup
...
}

# stack B
...
const jsonBlob = {
  securityGroupToUseInEMR: fgacSecurityGroup
}

Was there any resolution for this existing issue?

aws / aws-cdk