pulumi / pulumi-aws

An Amazon Web Services (AWS) Pulumi resource package, providing multi-language access to AWS
Apache License 2.0
455 stars 155 forks source link

RestApi won't update properly when using openapi #2321

Open tscully49 opened 1 year ago

tscully49 commented 1 year ago

What happened?

When updating resources using openapi spec on the RestApi resource, the config gets reverted back to the original-deployed config but the state still makes it look like the config was updated.

When digging into the terraform provider it looks like they have logic to try and ensure your RestApi config from resource inputs will overwrite the config from openapi, but I'm wondering if that logic is unintentionally rolling back the openapi config when it changes.

I am unable to figure out if this is a pulumi or terraform provider problem as I have only tried this out with pulumi so far.

Steps to reproduce

Use openapi to create a Rest API with a policy via pulumi.

Update the IP address in the policy of the openapi config and run another pulumi up.

See the policy has not changed.

Here is a sample pulumi code for a RestApi with openapi

const apigatewayBody = pulumi.output({
  openapi: '3.0.3',
  info: {
    title: 'test-api',
    version: '0.1.0',
  },
  paths: {
    '/test': {
      post: {
        responses: {
          200: {
            description: '200 response',
            content: {
              'application/json': {
                schema: {
                  $ref: '#/components/schemas/SuccesfulReturn',
                },
              },
            },
          },
          500: {
            description: 'Internal Server Error',
            content: {
              'application/json': {
                schema: {
                  $ref: '#/components/schemas/ErrorModel',
                },
              },
            },
          },

        },
        'x-amazon-apigateway-integration': {
          httpMethod: 'POST',
          uri: '{MY LAMBDA ARN}',
          responses: {
            default: {
              statusCode: '200',
            },
          },
          passthroughBehavior: 'when_no_match',
          contentHandling: 'CONVERT_TO_TEXT',
          type: 'aws_proxy',
        },
      },
    },
  },
  components: {
    schemas: {
      SuccesfulReturn: {
        type: 'object',
        required: [
          'message',
          'body',
          'statusCode',
        ],
        properties: {
          message: {
            type: 'string',
          },
          body: {
            type: 'string',
          },
          statusCode: {
            type: 'integer',
            minimum: 200,
            maximum: 300,
            default: 200,
          },
        },
      },
      ErrorModel: {
        type: 'object',
        required: [
          'message',
          'statusCode',
        ],
        properties: {
          message: {
            type: 'string',
          },
          statusCode: {
            type: 'integer',
            minimum: 100,
            maximum: 600,
            default: 500,
          },
        },
      },
    },
  },
  'x-amazon-apigateway-policy': pulumi.jsonParse(aws.iam.getPolicyDocumentOutput({
    statements: [
      {
        effect: 'Deny',
        principals: [{
          identifiers: ['*'],
          type: 'AWS',
        }],
        actions: ['execute-api:Invoke'],
        resources: ['execute-api:/*/POST/*'],
        conditions: [
          {
            // TODO: Protect against what VPCs can hit this as well
            test: 'NotIpAddress',
            variable: 'aws:SourceIp',
            values: [
              '10.0.0.2', // test
            ],
          },
        ],
      },
      {
        effect: 'Allow',
        principals: [{
          identifiers: ['*'],
          type: 'AWS',
        }],
        actions: ['execute-api:Invoke'],
        resources: ['execute-api:/*/POST/*'],
      },
    ],
  }).json),
})

new aws.apigateway.RestApi('test-api', {
  body: pulumi.jsonStringify(apigatewayBody),
  endpointConfiguration: {
    types: 'EDGE',
  },
  name: apigatewayBody.info.title,
})

Expected Behavior

I expect the openapi config to change parts of the RestApi that are not configured at the resource level.

Actual Behavior

The RestApi config is not updated in AWS.

Output of pulumi about

CLI Version 3.50.0 Go Version go1.19.4 Go Compiler gc

Plugins NAME VERSION aws 5.26.0 aws-apigateway 1.0.1 nodejs unknown

Host OS darwin Version 12.6.1 Arch arm64

This project is written in nodejs: executable='/Users/me/.asdf/shims/node' version='v16.19.0'

Dependencies: NAME VERSION @pulumi/aws 5.26.0 @pulumi/aws-apigateway 1.0.1 @pulumi/pulumi 3.51.0 @types/jest 29.2.5 @types/node 14.18.36 eslint 8.31.0 jest 29.3.1 ts-jest 28.0.8 typescript 4.9.4

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

t0yv0 commented 2 months ago

Thank you for filing this issue! I think I can reproduce with the following program:


import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const config = new pulumi.Config();

const step = config.requireNumber("step");

const apigatewayBody = pulumi.output({
  openapi: '3.0.3',
  info: {
    title: 'test-api',
    version: '0.1.0',
  },
  paths: {
    '/test': {
      post: {
        responses: {
          200: {
            description: '200 response',
            content: {
              'application/json': {
                schema: {
                  $ref: '#/components/schemas/SuccesfulReturn',
                },
              },
            },
          },
          500: {
            description: 'Internal Server Error',
            content: {
              'application/json': {
                schema: {
                  $ref: '#/components/schemas/ErrorModel',
                },
              },
            },
          },

        },
        // 'x-amazon-apigateway-integration': {
        //   httpMethod: 'POST',
        //   uri: '{MY LAMBDA ARN}',
        //   responses: {
        //     default: {
        //       statusCode: '200',
        //     },
        //   },
        //   passthroughBehavior: 'when_no_match',
        //   contentHandling: 'CONVERT_TO_TEXT',
        //   type: 'aws_proxy',
        // },
      },
    },
  },
  components: {
    schemas: {
      SuccesfulReturn: {
        type: 'object',
        required: [
          'message',
          'body',
          'statusCode',
        ],
        properties: {
          message: {
            type: 'string',
          },
          body: {
            type: 'string',
          },
          statusCode: {
            type: 'integer',
            minimum: 200,
            maximum: 300,
            default: 200,
          },
        },
      },
      ErrorModel: {
        type: 'object',
        required: [
          'message',
          'statusCode',
        ],
        properties: {
          message: {
            type: 'string',
          },
          statusCode: {
            type: 'integer',
            minimum: 100,
            maximum: 600,
            default: 500,
          },
        },
      },
    },
  },
  'x-amazon-apigateway-policy': pulumi.jsonParse(aws.iam.getPolicyDocumentOutput({
    statements: [
      {
        effect: 'Deny',
        principals: [{
          identifiers: ['*'],
          type: 'AWS',
        }],
        actions: ['execute-api:Invoke'],
        resources: ['execute-api:/*/POST/*'],
        conditions: [
          {
            // TODO: Protect against what VPCs can hit this as well
            test: 'NotIpAddress',
            variable: 'aws:SourceIp',
            values: [
                step===0?'10.0.0.2':'10.0.0.3', // test
            ],
          },
        ],
      },
      {
        effect: 'Allow',
        principals: [{
          identifiers: ['*'],
          type: 'AWS',
        }],
        actions: ['execute-api:Invoke'],
        resources: ['execute-api:/*/POST/*'],
      },
    ],
  }).json),
})

const ra = new aws.apigateway.RestApi('test-api', {
  body: pulumi.jsonStringify(apigatewayBody),
  endpointConfiguration: {
    types: 'EDGE',
  },
  name: apigatewayBody.info.title,
})

export const apiid = ra.id;
#!/usr/bin/env bash
set -euo pipefail
pulumi config set step 0
pulumi up --yes --skip-preview
pulumi config set step 1
pulumi up --yes --skip-preview

and check.sh:

#!/usr/bin/env bash

set -euo pipefail

aid=$(pulumi stack output apiid)

aws apigateway --region=us-east-1 get-rest-api --rest-api-id "$aid" |
    jq '.policy'

After Pulumi executes the update at step=1, the results of check.sh are consistent with the state of policy, and retains 10.0.0.2 from step=0.

"outputs": {
                    "policy": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Deny\",\"Principal\":{\"AWS\":\"*\"},\"Action\":\"execute-api:Invoke\",\"Resource\":\"arn:aws:execute-api:us-east-1:616138583583:oluujnt522/*/POST/*\",\"Condition\":{\"NotIpAddress\":{\"aws:SourceIp\":\"10.0.0.2\"}}},{\"Effect\":\"Allow\",\"Principal\":{\"AWS\":\"*\"},\"Action\":\"execute-api:Invoke\",\"Resource\":\"arn:aws:execute-api:us-east-1:616138583583:oluujnt522/*/POST/*\"}]}",
}

Even though the body output gets updated to 10.0.0.3:

                    "body": "{\"openapi\":\"3.0.3\",\"info\":{\"title\":\"test-api\",\"version\":\"0.1.0\"},\"paths\":{\"/test\":{\"post\":{\"responses\":{\"200\":{\"description\":\"200 response\",\"content\":{\"application/json\":{\"schema\":{\"$ref\":\"#/components/schemas/SuccesfulReturn\"}}}},\"500\":{\"description\":\"Internal Server Error\",\"content\":{\"application/json\":{\"schema\":{\"$ref\":\"#/components/schemas/ErrorModel\"}}}}}}}},\"components\":{\"schemas\":{\"SuccesfulReturn\":{\"type\":\"object\",\"required\":[\"message\",\"body\",\"statusCode\"],\"properties\":{\"message\":{\"type\":\"string\"},\"body\":{\"type\":\"string\"},\"statusCode\":{\"type\":\"integer\",\"minimum\":200,\"maximum\":300,\"default\":200}}},\"ErrorModel\":{\"type\":\"object\",\"required\":[\"message\",\"statusCode\"],\"properties\":{\"message\":{\"type\":\"string\"},\"statusCode\":{\"type\":\"integer\",\"minimum\":100,\"maximum\":600,\"default\":500}}}}},\"x-amazon-apigateway-policy\":{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Deny\",\"Action\":\"execute-api:Invoke\",\"Resource\":\"execute-api:/*/POST/*\",\"Principal\":{\"AWS\":\"*\"},\"Condition\":{\"NotIpAddress\":{\"aws:SourceIp\":\"10.0.0.3\"}}},{\"Effect\":\"Allow\",\"Action\":\"execute-api:Invoke\",\"Resource\":\"execute-api:/*/POST/*\",\"Principal\":{\"AWS\":\"*\"}}]}}",
t0yv0 commented 2 months ago

The same behavior holds upstream: https://github.com/hashicorp/terraform-provider-aws/issues/38515

t0yv0 commented 2 months ago

It is possible to apply the workaround of replaceOnChanges: ["body"] resource option to the RestAPI. This will cause a complete replacement of the resource. Let us know if this unblocks your scenario.