apigateway: Unable to import existing API Gateway with Stage (`Stage.from_stage_attributes` returns `__StageBaseProxy`)

fabian4cast commented 4 weeks ago

Describe the bug

Hello folks,

We previously deployed a Stack with an API Gateway via the Serverless framework, and decided to switch to CDK and hence rewrote the Stack configuration in CDK.

We managed to let CDK recognize most of the existing resources by overriding the logical IDs, e.g.

rest_api = aws_apigateway.RestApi(...)
rest_api.node.default_child.override_logical_id("<Logical ID of existing RestApi>")

This way CDK recognizes (or imports) the resources in the already existing Stack and updates them (if required) instead of deleting and re-creating. This works well for the RestApi. However, we also have a Stage that already exists for the RestApi, and doing the same with the existing Stage leads to the error Stage already exists. Here it seems that CDK is not able to recognize the existing Stage and just import it, but attempts to create a new one with the same name instead.

Hence, we attempted to just reference the existing Stage using aws_apigateway.Stage.from_stage_attributes as

rest_api = aws_apigateway.RestApi(
    scope=scope,
    id="RestApi",
    rest_api_name="name",
    deploy=False,
)
# Explicitly set logical ID to import existing RestApi previously deployed by Serverless
rest_api.node.default_child.override_logical_id("RestApi")

deployment = aws_apigateway.Deployment(
    scope=scope,
    id="Deployment",
    api=rest_api,
    stage_name="dev",
)

stage = aws_apigateway.Stage.from_stage_attributes(
    scope=scope,
    id="Stage",
    rest_api=rest_api,
    stage_name="dev",
)

# Since `aws_apigateway.RestApi` has `deploy=False`, we need to explicitly set this attribute.
rest_api.deployment_stage = stage

and this then fails because

TypeError: type of argument value must be aws_cdk.aws_apigateway.Stage; got aws_cdk.aws_apigateway._StageBaseProxy instead

According to the docs aws_apigateway.Stage.from_stage_attributes should return IStage, but apparently it seems to return _StageBaseProxy instead.

Thanks for the help in advance!

Moving from Serverless to CDK has generally been a great experience, at least from the point where we discovered the override_logical_id "hack", since that allowed us to import all previously existing resources that were part of the stacks, and didn't require cdk import.

Expected Behavior

aws_cdk.aws_apigateway.Stage.from_stage_attributes returns aws_cdk.aws_apigateway.IStage

Current Behavior

aws_cdk.aws_apigateway.Stage.from_stage_attributes returns aws_cdk.aws_apigateway._StageBaseProxy

Reproduction Steps

import aws_cdk
import aws_cdk.aws_apigateway as aws_apigateway
import constructs

class Stack(aws_cdk.Stack):
    def __init__(
        self,
        scope: constructs.Construct,
        **kwargs,
    ) -> None:
        super().__init__(
            scope=scope,
            id="Stack",
            **kwargs,
        )

        rest_api = aws_apigateway.RestApi(
            scope=self,
            id="RestApi",
            rest_api_name="name",
            deploy=False,
        )

        stage = aws_apigateway.Stage.from_stage_attributes(
            scope=self,
            id="Stage",
            rest_api=rest_api,
            stage_name="dev",
        )

        # Since `aws_apigateway.RestApi` has `deploy=False`, we need to explicitly set this attribute.
        rest_api.deployment_stage = stage

app = aws_cdk.App()

Stack(scope=app)

then cdk synth

Possible Solution

No response

Additional Information/Context

It is also worth mentioning that the Stage (and its corresponding Deployment) do currently not appear as a Resource of the Stack as of the state Serverless created! It just exists in the AWS API Gateway Console (or area, idk how to describe it). It seems that it was previously implicitly created by setting Stage attributes in Serverless, e.g.

RestApi:
  Type: AWS::ApiGateway::RestApi
  Properties:
    Name: "name"

Domain:
  Type: 'AWS::ApiGateway::DomainName'
  Properties:
    CertificateArn: "<arn>"
    DomainName: "<domain>"
    EndpointConfiguration:
      Types:
        - EDGE

ApiBasePathMapping:
  Type: 'AWS::ApiGateway::BasePathMapping'
  Properties:
    DomainName: !Ref Domain
    RestApiId: !Ref RestApi
    Stage: "dev"

CDK CLI Version

2.138.0 (build 6b41c8b)

Framework Version

No response

Node.js Version

v21.7.3

OS

Ubuntu 22.04 LTS

Language

Python

Language Version

3.12.0

Other information

Traceback (most recent call last):
  File "/home/user/engine/app.py", line 16, in <module>
    stack.Stack(
  File "/home/user/.cache/pypoetry/virtualenvs/engine-2aWaQfPc-py3.12/lib/python3.12/site-packages/jsii/_runtime.py", line 118, in __call__
    inst = super(JSIIMeta, cast(JSIIMeta, cls)).__call__(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/engine/stack.py", line 37, in __init__
    engine_api_gateway, engine_api_gateway_v1 = apigateway.create_resources(
                                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/engine/apigateway.py", line 58, in create_resources
    rest_api.deployment_stage = stage
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.cache/pypoetry/virtualenvs/engine-2aWaQfPc-py3.12/lib/python3.12/site-packages/aws_cdk/aws_apigateway/__init__.py", line 23101, in deployment_stage
    check_type(argname="argument value", value=value, expected_type=type_hints["value"])
  File "/home/user/.cache/pypoetry/virtualenvs/engine-2aWaQfPc-py3.12/lib/python3.12/site-packages/typeguard/__init__.py", line 785, in check_type
    raise TypeError(
TypeError: type of argument value must be aws_cdk.aws_apigateway.Stage; got aws_cdk.aws_apigateway._StageBaseProxy instead

ashishdhingra commented 4 weeks ago

@fabian4cast Good afternoon. I was able to reproduce the issue by:

Manually creating RestApi along with deployment stage (named BetaStage).

Using the below bare minimal code and running cdk synth:

from aws_cdk import (
Stack,
aws_apigateway as apigateway
)
from constructs import Construct

class PythonStack(Stack):

  def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
      super().__init__(scope, construct_id, **kwargs)

      restapi = apigateway.RestApi.from_rest_api_id(self, id="MyRestApi", rest_api_id="<<id-from-aws-console>>")
      stage = apigateway.Stage.from_stage_attributes(self, "MyStage", rest_api=restapi, stage_name="BetaStage")
      restapi.deployment_stage = stage

The reason you are getting the error is that apigateway.Stage.from_stage_attributes() returns IStage interface variable, whereas restapi.deployment_stage expects a concrete type Stage per below devompiled definition form VS Code:

@jsii.interface(jsii_type="aws-cdk-lib.aws_apigateway.IRestApi")
class IRestApi(_IResource_c80c4260, typing_extensions.Protocol):
    @builtins.property
    @jsii.member(jsii_name="restApiId")
    def rest_api_id(self) -> builtins.str:
        '''The ID of this API Gateway RestApi.

        :attribute: true
        '''
        ...
    @builtins.property
    @jsii.member(jsii_name="deploymentStage")
    def deployment_stage(self) -> "Stage":
        '''API Gateway stage that points to the latest deployment (if defined).'''
        ...

    @deployment_stage.setter
    def deployment_stage(self, value: "Stage") -> None:
        ...

Using the similar code in TypeScript reports same issue where IRestApi.deploymentStage is of type Stage.

I'm unsure about the workaround right now. Would review this with team.

Thanks, Ashish

pahud commented 4 weeks ago

Hi

The override_logical_id approach is an interesting hack that makes your CDK code to synthesize into exactly the same logical ID of the existing resource from an existing CFN stack deployed by serverless framework but this could lead to some other issues off the top of my head:

You will have the RestApi managed by CDK as L2 construct
other resources you import with fromXxx methods would not be the L2s nor L1s, instead they would be interfaces and you won't be allowed to use some methods only available for L2 constructs.
Your existing CFN stack would eventually have some resources like Stages or Methods not managed by CDK.

We do not have any public reference on migrating from serverless framework to CDK as there might be some different migration strategies, each has its pros and cons to consider.

I would suggest to reach out to AWS specialists to work together with your team to help you define the migration strategy. Feel free to ping me on cdk.dev if you need any further assistance.

github-actions[bot] commented 3 weeks ago

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

fabian4cast commented 3 weeks ago

Hey, thanks for the replies!

@ashishdhingra as of my experience it often is not so much of an issue whether a method returns e.g. Stage or IStage. Usually with from_xxx_attributes doesn't cause any issues (e.g. for iam.User, which we also use in the app).

@pahud

It's fine if the RestApi is an L2 construct. That is what we want.
It would be okay for some resources such as Stage here to just be in interface because
The Stage currently is already not part of the Stack managed by Serverless. It might be that Serverless can manage the Stage anyways since it somehow knows about it, but apparently it doesn't appear as a resource of the Stack in the CloudFormation console.

For this is, however, a blocker for us, but we are considering to find any other workaround that might help us for now.

aws / aws-cdk