search

pulumi / pulumi-aws

An Amazon Web Services (AWS) Pulumi resource package, providing multi-language access to AWS
Apache License 2.0
445 stars 154 forks source link

Constant logConfiguration diff without changes #4412

Closed quinnjr closed 6 days ago

quinnjr commented 2 weeks ago

Describe what happened

AWS Pipe keeps creating new diffs for logConfiguration even when no changes are made to the logConfiguration.

Sample program

const pipe = new aws.pipes.Pipe(
      generateNameTag('level-zero-pipe'),
      {
        name: generateNameTag('level-zero-pipe'),
        roleArn: pipeRole.arn,
        source: queues.levelZeroQueue!.arn,
        target: ecs.levelZeroCluster!.arn,
        sourceParameters: {
          sqsQueueParameters: {
            batchSize: 1
          }
        },
        targetParameters: {
          ecsTaskParameters: {
            taskDefinitionArn: ecs.levelZeroTaskDefinition!.arn,
            taskCount: 1,
            launchType: 'FARGATE',
            platformVersion: 'LATEST',
            networkConfiguration: {
              awsVpcConfiguration: {
                assignPublicIp: 'DISABLED',
                subnets: [vpc.privateSubnetZoneA.id, vpc.privateSubnetZoneB.id],
                securityGroups: [levelZeroSecurityGroup.id]
              }
            },
            overrides: {
              containerOverrides: [
                {
                  name: generateNameTag('ecs-level-zero-container'),
                  memory: ecs.containerDefinition.memory,
                  cpu: ecs.containerDefinition.cpu,
                  environments: [
                    {
                      name: 'SQS_MESSAGE',
                      value: '$.body'
                    }
                  ]
                }
              ]
            }
          },
          inputTemplate: JSON.stringify({
            containerOverrides: [
              {
                name: generateNameTag('ecs-level-zero-container'),
                environments: [
                  {
                    name: 'SQS_MESSAGE',
                    value: '<$.body>'
                  }
                ]
              }
            ]
          })
        },
        tags: {
          ...commonTag
        }
      },
      { dependsOn: [levelZeroEventRolePolicyAttatchment] }

Log output

~ ├─ aws:pipes:Pipe rb-staging-oregon-level-zero-pipe update [diff: -logConfiguration~logConfiguration,targetParamet

Affected Resource(s)

No response

Output of pulumi about

CLI          
Version      3.129.0
Go Version   go1.22.6
Go Compiler  gc

Plugins
KIND      NAME     VERSION
resource  aws      6.45.0
resource  aws      6.39.0
resource  aws      6.29.1
resource  aws      5.43.0
resource  awsx     2.13.0
resource  awsx     2.11.0
resource  awsx     1.0.6
resource  docker   4.5.4
resource  docker   4.5.4
resource  docker   3.6.1
resource  docker   3.6.1
resource  docker   3.6.1
resource  grafana  0.4.2
resource  grafana  0.4.2
language  nodejs   unknown
resource  random   4.16.3
resource  random   4.16.3

Host     
OS       ubuntu
Version  22.04
Arch     x86_64

This project is written in nodejs: executable='/home/joseph/.local/share/nvm/versions/node/v20.11.0/bin/node' version='v20.11.0'

Backend        
Name           pulumi.com
URL            https://app.pulumi.com/jquinn
User           jquinn
Organizations  jquinn, recordboss
Token type     personal

Dependencies:
NAME                              VERSION
eslint                            8.57.0
@typescript-eslint/eslint-plugin  7.16.1
@pulumi/awsx                      2.13.0
@pulumi/pulumi                    3.124.0
@typescript-eslint/parser         7.16.1
eslint-config-prettier            9.1.0
eslint-plugin-unicorn             52.0.0
ts-node                           10.9.2
eslint-plugin-prettier            5.2.1
@types/node                       20.14.11
@pulumiverse/grafana              v0.4.2
husky                             9.1.0
typescript                        5.5.3
@pulumi/aws                       6.45.0
@pulumi/random                    4.16.3
prettier                          3.3.3
lint-staged                       15.2.7
uuid                              9.0.1

Pulumi locates its logs in /tmp by default
warning: Failed to get information about the current stack: No current stack

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

corymhall commented 2 weeks ago

@quinnjr thanks for reporting this issue! Can you let us know what version of the AWS provider is being used for this resource? Looking at your pulumi about output it looks like there are several different versions being used so I want to isolate the version for this resource. You can find the version by exporting the stack (pulumi stack export), finding the pipes resource and then looking at the provider field.

I tried to reproduce the issue on a recent version of aws and I could not reproduce it.

quinnjr commented 2 weeks ago 
            {
                "urn": "urn:pulumi:prod::rb-infrastructure::pulumi:providers:aws::default_6_50_1",
                "custom": true,
                "id": "39ee0013-a4b6-4493-a9d5-95dfc6f83f63",
                "type": "pulumi:providers:aws",
                "inputs": {
                    "__internal": {},
                    "region": "us-west-2",
                    "skipCredentialsValidation": "false",
                    "skipRegionValidation": "true",
                    "version": "6.50.1"
                },
                "outputs": {
                    "region": "us-west-2",
                    "skipCredentialsValidation": "false",
                    "skipRegionValidation": "true",
                    "version": "6.50.1"
                },
                "created": "2024-08-29T23:09:45.133497758Z",
                "modified": "2024-08-29T23:09:45.133497758Z"
            },
            {
                "urn": "urn:pulumi:prod::rb-infrastructure::aws:pipes/pipe:Pipe::rb-prod-oregon-level-zero-pipe",
                "custom": true,
                "id": "rb-prod-oregon-level-zero-pipe",
                "type": "aws:pipes/pipe:Pipe",
                "inputs": {
                    "__defaults": [
                        "description",
                        "desiredState"
                    ],
                    "description": "Managed by Terraform",
                    "desiredState": "RUNNING",
                    "name": "rb-prod-oregon-level-zero-pipe",
                    "roleArn": "arn:aws:iam::947723167413:role/rb-prod-oregon-level-zero-pipe-role-a686caf",
                    "source": "arn:aws:sqs:us-west-2:947723167413:rb-prod-oregon-level-zero.fifo",
                    "sourceParameters": {
                        "__defaults": [],
                        "sqsQueueParameters": {
                            "__defaults": [],
                            "batchSize": 1
                        }
                    },
                    "tags": {
                        "Environment": "Production",
                        "ManagedBy": "Pulumi"
                    },
                    "tagsAll": {
                        "Environment": "Production",
                        "ManagedBy": "Pulumi"
                    },
                    "target": "arn:aws:ecs:us-west-2:947723167413:cluster/rb-prod-oregon-ecs-level-zero-cluster",
                    "targetParameters": {
                        "__defaults": [],
                        "ecsTaskParameters": {
                            "__defaults": [],
                            "launchType": "FARGATE",
                            "networkConfiguration": {
                                "__defaults": [],
                                "awsVpcConfiguration": {
                                    "__defaults": [],
                                    "assignPublicIp": "DISABLED",
                                    "securityGroups": [
                                        "sg-0d3045f99d7658391"
                                    ],
                                    "subnets": [
                                        "subnet-0dcf31dd05aad18dd",
                                        "subnet-00a45a5825e938be7"
                                    ]
                                }
                            },
                            "overrides": {
                                "__defaults": [],
                                "containerOverrides": [
                                    {
                                        "__defaults": [],
                                        "cpu": 2048,
                                        "environments": [
                                            {
                                                "__defaults": [],
                                                "name": "SQS_MESSAGE",
                                                "value": "$.body"
                                            }
                                        ],
                                        "memory": 8192,
                                        "name": "rb-prod-oregon-ecs-level-zero-container"
                                    }
                                ]
                            },
                            "platformVersion": "LATEST",
                            "taskCount": 1,
                            "taskDefinitionArn": "arn:aws:ecs:us-west-2:947723167413:task-definition/rb-prod-oregon-ecs-level-zero:12"
                        },
                        "inputTemplate": "{\"containerOverrides\":[{\"name\":\"rb-prod-oregon-ecs-level-zero-container\",\"environments\":[{\"name\":\"SQS_MESSAGE\",\"value\":\"\u003c$.body\u003e\"}]}]}"
                    }
                },
                "outputs": {
                    "__meta": "{\"e2bfb730-ecaa-11e6-8f88-34363bc7c4c0\":{\"create\":1800000000000,\"delete\":1800000000000,\"update\":1800000000000}}",
                    "arn": "arn:aws:pipes:us-west-2:947723167413:pipe/rb-prod-oregon-level-zero-pipe",
                    "description": "Managed by Terraform",
                    "desiredState": "RUNNING",
                    "enrichment": "",
                    "enrichmentParameters": null,
                    "id": "rb-prod-oregon-level-zero-pipe",
                    "logConfiguration": null,
                    "name": "rb-prod-oregon-level-zero-pipe",
                    "namePrefix": "",
                    "roleArn": "arn:aws:iam::947723167413:role/rb-prod-oregon-level-zero-pipe-role-a686caf",
                    "source": "arn:aws:sqs:us-west-2:947723167413:rb-prod-oregon-level-zero.fifo",
                    "sourceParameters": {
                        "activemqBrokerParameters": null,
                        "dynamodbStreamParameters": null,
                        "filterCriteria": null,
                        "kinesisStreamParameters": null,
                        "managedStreamingKafkaParameters": null,
                        "rabbitmqBrokerParameters": null,
                        "selfManagedKafkaParameters": null,
                        "sqsQueueParameters": {
                            "batchSize": 1,
                            "maximumBatchingWindowInSeconds": 0
                        }
                    },
                    "tags": {
                        "Environment": "Production",
                        "ManagedBy": "Pulumi"
                    },
                    "tagsAll": {
                        "Environment": "Production",
                        "ManagedBy": "Pulumi"
                    },
                    "target": "arn:aws:ecs:us-west-2:947723167413:cluster/rb-prod-oregon-ecs-level-zero-cluster",
                    "targetParameters": {
                        "batchJobParameters": null,
                        "cloudwatchLogsParameters": null,
                        "ecsTaskParameters": {
                            "capacityProviderStrategies": [],
                            "enableEcsManagedTags": false,
                            "enableExecuteCommand": false,
                            "group": "",
                            "launchType": "FARGATE",
                            "networkConfiguration": {
                                "awsVpcConfiguration": {
                                    "assignPublicIp": "DISABLED",
                                    "securityGroups": [
                                        "sg-0d3045f99d7658391"
                                    ],
                                    "subnets": [
                                        "subnet-0dcf31dd05aad18dd",
                                        "subnet-00a45a5825e938be7"
                                    ]
                                }
                            },
                            "overrides": {
                                "containerOverrides": [
                                    {
                                        "commands": [],
                                        "cpu": 2048,
                                        "environmentFiles": [],
                                        "environments": [
                                            {
                                                "name": "SQS_MESSAGE",
                                                "value": "$.body"
                                            }
                                        ],
                                        "memory": 8192,
                                        "memoryReservation": 0,
                                        "name": "rb-prod-oregon-ecs-level-zero-container",
                                        "resourceRequirements": []
                                    }
                                ],
                                "cpu": "",
                                "ephemeralStorage": null,
                                "executionRoleArn": "",
                                "inferenceAcceleratorOverrides": [],
                                "memory": "",
                                "taskRoleArn": ""
                            },
                            "placementConstraints": [],
                            "placementStrategies": [],
                            "platformVersion": "LATEST",
                            "propagateTags": "",
                            "referenceId": "",
                            "tags": {},
                            "taskCount": 1,
                            "taskDefinitionArn": "arn:aws:ecs:us-west-2:947723167413:task-definition/rb-prod-oregon-ecs-level-zero:12"
                        },
                        "eventbridgeEventBusParameters": null,
                        "httpParameters": null,
                        "inputTemplate": "{\"containerOverrides\":[{\"name\":\"rb-prod-oregon-ecs-level-zero-container\",\"environments\":[{\"name\":\"SQS_MESSAGE\",\"value\":\"\u003c$.body\u003e\"}]}]}",
                        "kinesisStreamParameters": null,
                        "lambdaFunctionParameters": null,
                        "redshiftDataParameters": null,
                        "sagemakerPipelineParameters": null,
                        "sqsQueueParameters": null,
                        "stepFunctionStateMachineParameters": null
                    }
                },
                "parent": "urn:pulumi:prod::rb-infrastructure::pulumi:pulumi:Stack::rb-infrastructure-prod",
                "dependencies": [
                    "urn:pulumi:prod::rb-infrastructure::aws:ec2/securityGroup:SecurityGroup::rb-prod-oregon-level-zero-sg",
                    "urn:pulumi:prod::rb-infrastructure::aws:iam/rolePolicyAttachment:RolePolicyAttachment::rb-prod-oregon-event-role-policy-attatchment-level-zero",
                    "urn:pulumi:prod::rb-infrastructure::aws:iam/role:Role::rb-prod-oregon-level-zero-pipe-role",
                    "urn:pulumi:prod::rb-infrastructure::aws:sqs/queue:Queue::rb-prod-oregon-sqs-level-zero",
                    "urn:pulumi:prod::rb-infrastructure::aws:ecs/cluster:Cluster::rb-prod-oregon-ecs-level-zero-cluster",
                    "urn:pulumi:prod::rb-infrastructure::aws:ecs/taskDefinition:TaskDefinition::rb-prod-oregon-ecs-level-zero-task-definition",
                    "urn:pulumi:prod::rb-infrastructure::aws:ec2/subnet:Subnet::rb-prod-oregon-subnet-private-a",
                    "urn:pulumi:prod::rb-infrastructure::aws:ec2/subnet:Subnet::rb-prod-oregon-subnet-private-b"
                ],
                "provider": "urn:pulumi:prod::rb-infrastructure::pulumi:providers:aws::default_6_50_1::39ee0013-a4b6-4493-a9d5-95dfc6f83f63",
                "propertyDependencies": {
                    "name": [],
                    "roleArn": [
                        "urn:pulumi:prod::rb-infrastructure::aws:iam/role:Role::rb-prod-oregon-level-zero-pipe-role"
                    ],
                    "source": [
                        "urn:pulumi:prod::rb-infrastructure::aws:sqs/queue:Queue::rb-prod-oregon-sqs-level-zero"
                    ],
                    "sourceParameters": [],
                    "tags": [],
                    "target": [
                        "urn:pulumi:prod::rb-infrastructure::aws:ecs/cluster:Cluster::rb-prod-oregon-ecs-level-zero-cluster"
                    ],
                    "targetParameters": [
                        "urn:pulumi:prod::rb-infrastructure::aws:ecs/taskDefinition:TaskDefinition::rb-prod-oregon-ecs-level-zero-task-definition",
                        "urn:pulumi:prod::rb-infrastructure::aws:ec2/subnet:Subnet::rb-prod-oregon-subnet-private-a",
                        "urn:pulumi:prod::rb-infrastructure::aws:ec2/subnet:Subnet::rb-prod-oregon-subnet-private-b",
                        "urn:pulumi:prod::rb-infrastructure::aws:ec2/securityGroup:SecurityGroup::rb-prod-oregon-level-zero-sg"
                    ]
                },
                "created": "2024-06-15T16:32:17.701027279Z",
                "modified": "2024-08-14T17:37:51.632328857Z",
                "sourcePosition": "project:///file:/home/joseph/rb-infrastructure/lib/eventbridge.ts#122,22"
            }
t0yv0 commented 1 week ago

Unfortunately still cannot repro this, can you please provide a complete minimal program and sequence of steps?

I tried an example program but repeat pulumi up is OK in this case.

This snippet:

logConfiguration~logConfiguration,targetParamet

Makes me think maybe something is also being picked up as changing in targetParameters in your case?

Program I tried:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const main = aws.getCallerIdentity({});

const accountId: pulumi.Output<string> = pulumi.output(main.then(m => m.accountId));

export const arp: aws.iam.PolicyDocument = {
    Version: "2012-10-17",
    Statement: [{
        Effect: "Allow",
        Action: "sts:AssumeRole",
        Principal: {
            Service: "pipes.amazonaws.com",
        },
        Condition: {
            StringEquals: {
                "aws:SourceAccount": accountId,
            },
        },
    }],
};

const example = new aws.iam.Role("example", {assumeRolePolicy: arp});

const sourceQueue = new aws.sqs.Queue("source", {});

const source = new aws.iam.RolePolicy("source", {
    role: example.id,
    policy: pulumi.jsonStringify({
        Version: "2012-10-17",
        Statement: [{
            Effect: "Allow",
            Action: [
                "sqs:DeleteMessage",
                "sqs:GetQueueAttributes",
                "sqs:ReceiveMessage",
            ],
            Resource: [sourceQueue.arn],
        }],
    }),
});

const targetQueue = new aws.sqs.Queue("target", {});

const target = new aws.iam.RolePolicy("target", {
    role: example.id,
    policy: pulumi.jsonStringify({
        Version: "2012-10-17",
        Statement: [{
            Effect: "Allow",
            Action: ["sqs:SendMessage"],
            Resource: [targetQueue.arn],
        }],
    }),
});

const examplePipe = new aws.pipes.Pipe("example", {
    name: "example-pipe",
    roleArn: example.arn,
    source: sourceQueue.arn,
    target: targetQueue.arn,
}, {
    dependsOn: [
        source,
        target,
    ],
});
quinnjr commented 1 week ago 
import * as aws from '@pulumi/aws';

import { generateNameTag } from './helpers/generate-name-tag';
import { commonTag } from './helpers/common-tag';
import { VPCExport } from './vpc';
import { QueueExport } from './sqs';
import { ECSExport } from './ecs';

export const createEventBridge = async (
  stackName: string,
  vpc: VPCExport,
  queues: QueueExport,
  ecs: ECSExport
) => {
  if (stackName !== 'infra') {
    const levelZeroEventRole = new aws.iam.Role(
      generateNameTag('event-role-level-zero'),
      {
        assumeRolePolicy: aws.iam.assumeRolePolicyForPrincipal({
          Service: 'events.amazonaws.com'
        }),
        tags: {
          ...commonTag
        }
      }
    );

    const levelZeroEventRolePolicyAttatchment =
      new aws.iam.RolePolicyAttachment(
        generateNameTag('event-role-policy-attatchment-level-zero'),
        {
          role: levelZeroEventRole,
          policyArn:
            'arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceEventsRole'
        }
      );

    const pipeRole = new aws.iam.Role(generateNameTag('level-zero-pipe-role'), {
      assumeRolePolicy: aws.iam.assumeRolePolicyForPrincipal({
        Service: 'pipes.amazonaws.com'
      })
    });

    const pipeRolePolicy = new aws.iam.RolePolicy(
      generateNameTag('level-zero-pipe-role-policy'),
      {
        role: pipeRole.id,
        policy: {
          Version: '2012-10-17',
          Statement: [
            {
              Effect: 'Allow',
              Action: [
                'events:PutEvents',
                'sqs:SendMessage',
                'sqs:GetQueueAttributes',
                'sqs:ReceiveMessage',
                'sqs:DeleteMessage'
              ],
              Resource: queues.levelZeroQueue!.arn
            },
            {
              Effect: 'Allow',
              Action: ['ecs:RunTask'],
              Resource: ecs.levelZeroTaskDefinition!.arn,
              Condition: {
                ArnEquals: {
                  'ecs:cluster': ecs.levelZeroCluster!.arn
                }
              }
            },
            {
              Effect: 'Allow',
              Action: ['iam:PassRole'],
              Resource: '*'
            }
          ]
        }
      }
    );

    const levelZeroSecurityGroup = new aws.ec2.SecurityGroup(
      generateNameTag('level-zero-sg'),
      {
        vpcId: vpc.vpc.id,
        ingress: [
          {
            protocol: 'tcp',
            fromPort: 80,
            toPort: 80,
            cidrBlocks: ['0.0.0.0/0']
          },
          {
            protocol: 'tcp',
            fromPort: 443,
            toPort: 443,
            cidrBlocks: ['0.0.0.0/0']
          },
          {
            protocol: 'tcp',
            fromPort: 3306,
            toPort: 3306,
            cidrBlocks: ['10.0.0.0/16', '10.1.0.0/16', '10.2.0.0/16']
          },
          {
            protocol: 'tcp',
            fromPort: 5432,
            toPort: 5432,
            cidrBlocks: ['10.0.0.0/16', '10.1.0.0/16', '10.2.0.0/16']
          }
        ],
        egress: [
          {
            protocol: 'tcp',
            fromPort: 80,
            toPort: 80,
            cidrBlocks: ['0.0.0.0/0']
          },
          {
            protocol: 'tcp',
            fromPort: 443,
            toPort: 443,
            cidrBlocks: ['0.0.0.0/0']
          },
          {
            protocol: 'tcp',
            fromPort: 3306,
            toPort: 3306,
            cidrBlocks: ['10.0.0.0/16', '10.1.0.0/16', '10.2.0.0/16']
          },
          {
            protocol: 'tcp',
            fromPort: 5432,
            toPort: 5432,
            cidrBlocks: ['10.0.0.0/16', '10.1.0.0/16', '10.2.0.0/16']
          }
        ],
        tags: {
          ...commonTag
        }
      }
    );

    const logGroup = new aws.cloudwatch.LogGroup(
      generateNameTag('level-zero-pipe-logs'),
      {
        name: generateNameTag('level-zero-pipe-logs'),
        retentionInDays: 30,
        tags: {
          ...commonTag
        }
      }
    );

    const pipe = new aws.pipes.Pipe(
      generateNameTag('level-zero-pipe'),
      {
        name: generateNameTag('level-zero-pipe'),
        roleArn: pipeRole.arn,
        source: queues.levelZeroQueue!.arn,
        target: ecs.levelZeroCluster!.arn,
        sourceParameters: {
          sqsQueueParameters: {
            batchSize: 1
          }
        },
        targetParameters: {
          ecsTaskParameters: {
            taskDefinitionArn: ecs.levelZeroTaskDefinition!.arn,
            taskCount: 1,
            launchType: 'FARGATE',
            platformVersion: 'LATEST',
            networkConfiguration: {
              awsVpcConfiguration: {
                assignPublicIp: 'DISABLED',
                subnets: [vpc.privateSubnetZoneA.id, vpc.privateSubnetZoneB.id],
                securityGroups: [levelZeroSecurityGroup.id]
              }
            },
            overrides: {
              containerOverrides: [
                {
                  name: generateNameTag('ecs-level-zero-container'),
                  memory: ecs.containerDefinition.memory,
                  cpu: ecs.containerDefinition.cpu,
                  environments: [
                    {
                      name: 'SQS_MESSAGE',
                      value: '$.body'
                    }
                  ]
                }
              ]
            }
          },
          inputTemplate: JSON.stringify({
            containerOverrides: [
              {
                name: generateNameTag('ecs-level-zero-container'),
                environments: [
                  {
                    name: 'SQS_MESSAGE',
                    value: '<$.body>'
                  }
                ]
              }
            ]
          })
        },
        tags: {
          ...commonTag
        }
      },
      { dependsOn: [levelZeroEventRolePolicyAttatchment] }
    );
  }
};
t0yv0 commented 1 week ago

Thanks for your input, unfortunately this is still not a repro as it has some dangling references. I've taken some liberties and tried the following, but it does not repro:


import * as aws from '@pulumi/aws';
import * as awsx from '@pulumi/awsx';

// import { generateNameTag } from './helpers/generate-name-tag';
// import { commonTag } from './helpers/common-tag';
// import { VPCExport } from './vpc';
// import { QueueExport } from './sqs';
// import { ECSExport } from './ecs';

function generateNameTag(x: string): string {
    return x;
}

const vpc = new awsx.ec2.Vpc("myvpc", {});

const levelZeroEventRole = new aws.iam.Role(
    generateNameTag('event-role-level-zero'),
    {
        assumeRolePolicy: aws.iam.assumeRolePolicyForPrincipal({
            Service: 'events.amazonaws.com'
        }),
    }
);

const levelZeroEventRolePolicyAttatchment =
    new aws.iam.RolePolicyAttachment(
        generateNameTag('event-role-policy-attatchment-level-zero'),
        {
            role: levelZeroEventRole,
            policyArn:
            'arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceEventsRole'
        }
    );

const pipeRole = new aws.iam.Role(generateNameTag('level-zero-pipe-role'), {
    assumeRolePolicy: aws.iam.assumeRolePolicyForPrincipal({
        Service: 'pipes.amazonaws.com'
    })
});

const source = new aws.sqs.Queue("queue", {});

const pipeRolePolicy = new aws.iam.RolePolicy(
    generateNameTag('level-zero-pipe-role-policy'),
    {
        role: pipeRole.id,
        policy: {
            Version: '2012-10-17',
            Statement: [
                {
                  Effect: 'Allow',
                  Action: [
                    'events:PutEvents',
                    'sqs:SendMessage',
                    'sqs:GetQueueAttributes',
                    'sqs:ReceiveMessage',
                    'sqs:DeleteMessage'
                  ],
                  Resource: source.arn
                },
                // {
                //   Effect: 'Allow',
                //   Action: ['ecs:RunTask'],
                //   Resource: ecs.levelZeroTaskDefinition!.arn,
                //   Condition: {
                //     ArnEquals: {
                //       'ecs:cluster': ecs.levelZeroCluster!.arn
                //     }
                //   }
                // },
                {
                    Effect: 'Allow',
                    Action: ['iam:PassRole'],
                    Resource: '*'
                }
            ]
        }
    }
);

const levelZeroSecurityGroup = new aws.ec2.SecurityGroup(
    generateNameTag('level-zero-sg'),
    {
        vpcId: vpc.vpc.id,
        ingress: [
            {
                protocol: 'tcp',
                fromPort: 80,
                toPort: 80,
                cidrBlocks: ['0.0.0.0/0']
            },
            {
                protocol: 'tcp',
                fromPort: 443,
                toPort: 443,
                cidrBlocks: ['0.0.0.0/0']
            },
            {
                protocol: 'tcp',
                fromPort: 3306,
                toPort: 3306,
                cidrBlocks: ['10.0.0.0/16', '10.1.0.0/16', '10.2.0.0/16']
            },
            {
                protocol: 'tcp',
                fromPort: 5432,
                toPort: 5432,
                cidrBlocks: ['10.0.0.0/16', '10.1.0.0/16', '10.2.0.0/16']
            }
        ],
        egress: [
            {
                protocol: 'tcp',
                fromPort: 80,
                toPort: 80,
                cidrBlocks: ['0.0.0.0/0']
            },
            {
                protocol: 'tcp',
                fromPort: 443,
                toPort: 443,
                cidrBlocks: ['0.0.0.0/0']
            },
            {
                protocol: 'tcp',
                fromPort: 3306,
                toPort: 3306,
                cidrBlocks: ['10.0.0.0/16', '10.1.0.0/16', '10.2.0.0/16']
            },
            {
                protocol: 'tcp',
                fromPort: 5432,
                toPort: 5432,
                cidrBlocks: ['10.0.0.0/16', '10.1.0.0/16', '10.2.0.0/16']
            }
        ]
    }
);

const logGroup = new aws.cloudwatch.LogGroup(
    generateNameTag('level-zero-pipe-logs'),
    {
        name: generateNameTag('level-zero-pipe-logs'),
        retentionInDays: 30
    }
);

const target = new aws.ecs.Cluster("cluster1", {});

const taskdef = new aws.ecs.TaskDefinition("service", {
    family: "service",
    containerDefinitions: JSON.stringify([
        {
            name: "first",
            image: "service-first",
            cpu: 10,
            memory: 512,
            essential: true,
            portMappings: [{
                containerPort: 80,
                hostPort: 80,
            }],
        },
        {
            name: "second",
            image: "service-second",
            cpu: 10,
            memory: 256,
            essential: true,
            portMappings: [{
                containerPort: 443,
                hostPort: 443,
            }],
        },
    ]),
    volumes: [{
        name: "service-storage",
        hostPath: "/ecs/service-storage",
    }],
    placementConstraints: [{
        type: "memberOf",
        expression: "attribute:ecs.availability-zone in [us-west-2a, us-west-2b]",
    }],
});

const pipe = new aws.pipes.Pipe(
    generateNameTag('level-zero-pipe'),
    {
        name: generateNameTag('level-zero-pipe'),
        roleArn: pipeRole.arn,
        source: source.arn,
        target: target.arn,
        sourceParameters: {
            sqsQueueParameters: {
                batchSize: 1
            }
        },
        targetParameters: {
            ecsTaskParameters: {
                taskDefinitionArn: taskdef.arn,
                taskCount: 1,
                launchType: 'FARGATE',
                platformVersion: 'LATEST',
                networkConfiguration: {
                    awsVpcConfiguration: {
                        assignPublicIp: 'DISABLED',
                        subnets: vpc.subnets.apply(x => x.map(s => s.id)),
                        securityGroups: [levelZeroSecurityGroup.id]
                    }
                },
                overrides: {
                    containerOverrides: [
                        {
                            name: generateNameTag('ecs-level-zero-container'),
                            memory: 32,
                            cpu: 16,
                            environments: [
                                {
                                    name: 'SQS_MESSAGE',
                                    value: '$.body'
                                }
                            ]
                        }
                    ]
                }
            },
            inputTemplate: JSON.stringify({
                containerOverrides: [
                    {
                        name: generateNameTag('ecs-level-zero-container'),
                        environments: [
                            {
                                name: 'SQS_MESSAGE',
                                value: '<$.body>'
                            }
                        ]
                    }
                ]
            })
        },
    },
    { dependsOn: [levelZeroEventRolePolicyAttatchment] }
);