search

aws / copilot-cli

The AWS Copilot CLI is a tool for developers to build, release and operate production ready containerized applications on AWS App Runner or Amazon ECS on AWS Fargate.
https://aws.github.io/copilot-cli/
Apache License 2.0
3.54k stars 417 forks source link

How to get around "chown: operation not allowed" for mounted EFS? #4431

Closed songhobby closed 1 year ago

KollaAdithya commented 1 year ago

Hello @songhobby ! I have few questions that will help me in troubleshooting your issue

  1. Are you trying to create a managed EFS using Copilot Workload manifest.
  2. Can you provide me a bit more background on how you ended up with this error and what exactly is the use case that you are trying to achieve.
songhobby commented 1 year ago

Hi @KollaAdithya I use the docker image ‘ipfs/kubo’ and deploy it as a service using ‘copilot svc deploy’. The docker image use a volume ‘/data/ipfs’ and during the docker image building, it calls ‘chown ipfs:users /data/ipfs’ but this operation is not allowed. The service is run with the user ‘ipfs’

KollaAdithya commented 1 year ago

Hello @songhobby Can you please paste your manifest and dockerfile. This would be very helpful for us to further dig into your issue.

songhobby commented 1 year ago

@KollaAdithya 

# The manifest for the "graph-node" service.
# Read the full specification for the "Load Balanced Web Service" type at:
#  https://aws.github.io/copilot-cli/docs/manifest/lb-web-service/

# Your service name will be used in naming your resources like log groups, ECS services, etc.
name: graph-node
type: Load Balanced Web Service

# Distribute traffic to your service.
http:
  # Requests to this path will be forwarded to your service.
  # To match all requests you can use the "/" path.
  path: "/"
  # You can specify a custom health check path. The default is "/".
  healthcheck:
    path: "/"
    port: 8000
    success_codes: "200-399"

# Configuration for your containers and service.
image:
  # Docker build arguments. For additional overrides: https://aws.github.io/copilot-cli/docs/manifest/lb-web-service/#image-build
  build: docker/graph-node/Dockerfile
  # Port exposed through your container to route traffic to it.
  port: 8000

cpu: 1024 # Number of CPU units for the task.
memory: 4096 # Amount of memory in MiB used by the task.
# count: 1 # Number of tasks that should be running in your service.
count:
  range: 1-4
  cooldown:
    in: 30s
    out: 60s
  cpu_percentage: 70
  memory_percentage:
    value: 80
    cooldown:
      in: 80s
      out: 160s
  requests: 10000
  response_time: 2s
exec: true # Enable running commands in your container.
network:
  connect: true # Enable Service Connect for intra-environment traffic between services.

storage:
  volumes:
    ipfs:
      efs: true
      read_only: false
      path: /data/ipfs

# Optional fields for more advanced use-cases.
#
variables:
  postgres_db: graph-node
  ipfs: http://ipfs:5001
  GRAPH_LOG: info

#  GITHUB_TOKEN: GITHUB_TOKEN  # The key is the name of the environment variable, the value is the name of the SSM parameter.

# You can override any of the values defined above by environment.
#environments:
#  test:
#    count: 2               # Number of tasks to run for the "test" environment.
#    deployment:            # The deployment strategy for the "test" environment.
#       rolling: 'recreate' # Stops existing tasks before new ones are started for faster deployments.

environments:
  dev:
    variables:
      postgres_host: ""
      ethereum: ""
    secrets:
      postgres_user:
        secretsmanager: ""
      postgres_pass:
        secretsmanager: ""
songhobby commented 1 year ago

@KollaAdithya This is the Dockerfile

FROM graphprotocol/graph-node
songhobby commented 1 year ago

@KollaAdithya If I were to take out the 

storage:
  volumes:
    ipfs:
      efs: true
      read_only: false
      path: /data/ipfs

Everything works

huanjani commented 1 year ago

Hello, @songhobby!

I've been reading a bit about ipfs/kubo because I am unfamiliar with it, but I am still confused, sorry. You are running Kubo IPFS inside of Docker, right? I don't see where you are passing the ipfs/kubo image location to Copilot. Is that a different service than the one with the FROM graphprotocol/graph-node Dockerfile? When you say that without the storage section of the manifest everything works, what do you mean? Are you getting the results that you want? Have you read the various Copilot EFS options here?

Thanks!

songhobby commented 1 year ago

Hello, @songhobby!

I've been reading a bit about ipfs/kubo because I am unfamiliar with it, but I am still confused, sorry. You are running Kubo IPFS inside of Docker, right? I don't see where you are passing the ipfs/kubo image location to Copilot. Is that a different service than the one with the FROM graphprotocol/graph-node Dockerfile? When you say that without the storage section of the manifest everything works, what do you mean? Are you getting the results that you want? Have you read the various Copilot EFS options here?

Thanks!

It’s not complicated at all. I want to persist the ‘/data/ipfs’ so I need to use EFS by defining a storage entry. But the docker has the operation of ‘chown’, which isn’t allowed by AWS EFS I guess

huanjani commented 1 year ago

Closing this issue.

If you have Copilot-related questions, or can share more about your Copilot setup, please feel free to reopen!

I'm seeing a fair bit online re: users/permissions and EFS, so hopefully you'll find what you're looking for!

songhobby commented 1 year ago

Closing this issue.

If you have Copilot-related questions, or can share more about your Copilot setup, please feel free to reopen!

I'm seeing a fair bit online re: users/permissions and EFS, so hopefully you'll find what you're looking for!

I shared the entire Dockerfile and the copilot manifest. I had no problem deploying it using ECS cli but just couldn’t do it using copilot

huanjani commented 1 year ago

Gotcha. I think the issue may lie in the way that Copilot defines access points per service, which enforces user and group IDs. Service-managed EFS won't let non-owner users change ownership. You could try defining an EFS addon for only this service, with mount points in each Copilot subnet and an exported security group from the EFS filesystem. Something like:

Parameters:
  App:
    Type: String
  Env:
    Type: String
  Name:
    Type: String
Resources:
  FileSystem:
    Type: AWS::EFS::FileSystem
    Properties:
      Encrypted: true
      LifecyclePolicies:
        - TransitionToIA: AFTER_30_DAYS
      PerformanceMode: generalPurpose
      ThroughputMode: bursting
  EFSSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      VpcId: ${REPLACE_ME_VPC_ID}
  EFSSecurityGroupIngressFromSelf:
    Type: AWS::EC2::SecurityGroupIngress
    Properties:
      GroupId: !Ref EFSSecurityGroup
      IpProtocol: -1
      SourceSecurityGroupId: !Ref EFSSecurityGroup

  MountTarget1:
    Type: AWS::EFS::MountTarget
    Properties:
      FileSystemId: !Ref FileSystem
      SubnetId: ${REPLACE_ME_SUBNET_1_ID}
      SecurityGroups:
        - !Ref EFSSecurityGroup
  MountTarget2:
    Type: AWS::EFS::MountTarget
    Properties:
      FileSystemId: !Ref FileSystem
      SubnetId: ${REPLACE_ME_SUBNET_2_ID}
      SecurityGroups:
        - !Ref EFSSecurityGroup
Outputs:
  SecurityGroupId:
    Value: !GetAtt SecurityGroup.GroupId
huanjani commented 1 year ago

You'll need to change your workload manifest to something like

storage:
  volumes:
    myEFSVolume: # This is a variable key and can be set to an arbitrary string.
      path: '/etc/mount1'
      efs:
        id: fs-1234567

with the ID of the fs.

songhobby commented 1 year ago

@huanjani Thanks for the clarification. Is is possible to leverage the settings of uid and gid to give the service the root access? I'd prefer pure copilot configuration as it is more succinct. I tried something like the following but it doesn't work

storage:
  volumes:
    myManagedEFSVolume:
      efs: 
        uid: 0
        gid: 0
      path: /var/efs
      read_only: false
huanjani commented 1 year ago

@songhobby, I consulted our EFS guru again, and they suggested trying ☝🏼 with uid and gid again, because the values are set at creation. Delete the service from just that env using copilot svc delete -n graph-node -e dev [or whatever the name of the env you deployed to is] then redeploy. 🤞🏼 Not sure if this will work to give the service access; please let us know how it goes!

github-actions[bot] commented 1 year ago

This issue is stale because it has been open 60 days with no response activity, and is tagged with pending/question. Remove the stale label, add a comment, or this will be closed in 14 days.

github-actions[bot] commented 1 year ago

This issue is closed due to inactivity. Feel free to reopen the issue if you have any follow-ups!

songhobby commented 1 year ago

@huanjani Can you please reopen this issue?

@songhobby, I consulted our EFS guru again, and they suggested trying ☝🏼 with uid and gid again, because the values are set at creation. Delete the service from just that env using copilot svc delete -n graph-node -e dev [or whatever the name of the env you deployed to is] then redeploy. 🤞🏼 Not sure if this will work to give the service access; please let us know how it goes!

I tried using uid and gid and here is the error when deploying it.

validate manifest against environment "dev": validate "storage": validate "volumes[ipfs]": validate "efs": "uid" must not be 0

The manifest.yml is the following

storage:
  volumes:
    ipfs:
      efs:
        uid: 0
        gid: 0
      path: /data/ipfs
      read_only: false
huanjani commented 1 year ago

Gotcha.

Looks like Copilot has that manifest validation to prevent root access. You can set uid and gid to 0 within a CloudFormation addon, as referenced above. This resource will help, if you go that route https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-efs-filesystem.html.

Another (easier) option is to use our new yaml patch override feature! Your patch would look something like:

- op: add
  path: /Resources/AccessPoint/Properties/PosixUser/Uid
  value: 0
- op: add
  path: /Resources/AccessPoint/Properties/PosixUser/Gid
  value: 0

Follow the steps in the linked-to docs for setting up the yaml patch.

Hopefully those setting will allow chown for you.

songhobby commented 1 year ago

yaml patch override

I tried the override ops with the following but it still failed. Since I'm not familiar with how cloudformation works, I first assign uid and gid to non-zero numberd and replace them afterwards instead.

- op: replace
  path: /Resources/AccessPoint/Properties/PosixUser/Uid
  value: 0
- op: replace
  path: /Resources/AccessPoint/Properties/PosixUser/Gid
  value: 0
- op: replace
  path: /Resources/AccessPoint/Properties/RootDirectory/CreationInfo/OwnerUid
  value: 0
- op: replace
  path: /Resources/AccessPoint/Properties/RootDirectory/CreationInfo/OwnerGid
  value: 0
The overriden cloudformation yml file 
# Copyright Amazon.com Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: MIT-0
AWSTemplateFormatVersion: 2010-09-09
Description: CloudFormation template that represents a load balanced web service on Amazon ECS using AWS Copilot with YAML patches.
Metadata:
    Manifest: |
        # The manifest for the "ipfs" service.
        # Read the full specification for the "Load Balanced Web Service" type at:
        #  https://aws.github.io/copilot-cli/docs/manifest/lb-web-service/

        # Your service name will be used in naming your resources like log groups, ECS services, etc.
        name: ipfs
        type: Load Balanced Web Service

        # Distribute traffic to your service.
        http:
          # Requests to this path will be forwarded to your service.
          # To match all requests you can use the "/" path.
          path: "/"
          # You can specify a custom health check path. The default is "/".
          # healthcheck: '/'
          healthcheck:
            path: "/"
            port: 5001
            success_codes: "200-399"

        # Configuration for your containers and service.
        image:
          location: docker/ipfs/Dockerfile
          # Port exposed through your container to route traffic to it.
          port: 5001

        cpu: 1024 # Number of CPU units for the task.
        memory: 2048 # Amount of memory in MiB used by the task.
        # count: 1 # Number of tasks that should be running in your service.
        count:
          range: 1-4
          cooldown:
            in: 30s
            out: 60s
          cpu_percentage: 70
          memory_percentage:
            value: 80
            cooldown:
              in: 80s
              out: 160s
          requests: 10000
          response_time: 2s
        exec: true # Enable running commands in your container.
        network:
          connect: true # Enable Service Connect for intra-environment traffic between services.

        # storage:
        # readonly_fs: true       # Limit to read-only access to mounted root filesystems.
        storage:
          volumes:
            ipfs:
              efs:
                uid: 123456
                gid: 123456
              path: /data/ipfs
              read_only: false
        # Optional fields for more advanced use-cases.
        #
        #variables:                    # Pass environment variables as key value pairs.
        #  LOG_LEVEL: info

        #secrets:                      # Pass secrets from AWS Systems Manager (SSM) Parameter Store.
        #  GITHUB_TOKEN: GITHUB_TOKEN  # The key is the name of the environment variable, the value is the name of the SSM parameter.

        # You can override any of the values defined above by environment.
        #environments:
        #  test:
        #    count: 2               # Number of tasks to run for the "test" environment.
        #    deployment:            # The deployment strategy for the "test" environment.
        #       rolling: 'recreate' # Stops existing tasks before new ones are started for faster deployments.
Parameters:
    AppName:
        Type: String
    EnvName:
        Type: String
    WorkloadName:
        Type: String
    ContainerImage:
        Type: String
    ContainerPort:
        Type: Number
    TaskCPU:
        Type: String
    TaskMemory:
        Type: String
    TaskCount:
        Type: Number
    DNSDelegated:
        Type: String
        AllowedValues: [true, false]
    LogRetention:
        Type: Number
    AddonsTemplateURL:
        Description: 'URL of the addons nested stack template within the S3 bucket.'
        Type: String
        Default: ""
    EnvFileARN:
        Description: 'URL of the environment file.'
        Type: String
        Default: ""
    TargetContainer:
        Type: String
    TargetPort:
        Type: Number
    HTTPSEnabled:
        Type: String
        AllowedValues: [true, false]
    RulePath:
        Type: String
Conditions:
    IsGovCloud: !Equals [!Ref "AWS::Partition", "aws-us-gov"]
    HasAssociatedDomain: !Equals [!Ref DNSDelegated, true]
    HasAddons: !Not [!Equals [!Ref AddonsTemplateURL, ""]]
    HasEnvFile: !Not [!Equals [!Ref EnvFileARN, ""]]
Resources: # If a bucket URL is specified, that means the template exists.
    LogGroup:
        Metadata:
            'aws:copilot:description': 'A CloudWatch log group to hold your service logs'
        Type: AWS::Logs::LogGroup
        Properties:
            LogGroupName: !Join ['', [/copilot/, !Ref AppName, '-', !Ref EnvName, '-', !Ref WorkloadName]]
            RetentionInDays: !Ref LogRetention
    TaskDefinition:
        Metadata:
            'aws:copilot:description': 'An ECS task definition to group your containers and run them on ECS'
        Type: AWS::ECS::TaskDefinition
        DependsOn: LogGroup
        Properties:
            Family: !Join ['', [!Ref AppName, '-', !Ref EnvName, '-', !Ref WorkloadName]]
            NetworkMode: awsvpc
            RequiresCompatibilities:
                - FARGATE
            Cpu: !Ref TaskCPU
            Memory: !Ref TaskMemory
            ExecutionRoleArn: !GetAtt ExecutionRole.Arn
            TaskRoleArn: !GetAtt TaskRole.Arn
            ContainerDefinitions:
                - Name: !Ref WorkloadName
                  Image: !Ref ContainerImage
                  Environment:
                    - Name: COPILOT_APPLICATION_NAME
                      Value: !Sub '${AppName}'
                    - Name: COPILOT_SERVICE_DISCOVERY_ENDPOINT
                      Value: dev.ao.local
                    - Name: COPILOT_ENVIRONMENT_NAME
                      Value: !Sub '${EnvName}'
                    - Name: COPILOT_SERVICE_NAME
                      Value: !Sub '${WorkloadName}'
                    - Name: COPILOT_LB_DNS
                      Value: !GetAtt EnvControllerAction.PublicLoadBalancerDNSName
                    - Name: COPILOT_MOUNT_POINTS
                      Value: '{"ipfs":"/data/ipfs"}'
                  EnvironmentFiles:
                    - !If
                      - HasEnvFile
                      - Type: s3
                        Value: !Ref EnvFileARN
                      - !Ref AWS::NoValue
                  LogConfiguration:
                    LogDriver: awslogs
                    Options:
                        awslogs-region: !Ref AWS::Region
                        awslogs-group: !Ref LogGroup
                        awslogs-stream-prefix: copilot
                  MountPoints:
                    - ContainerPath: '/data/ipfs'
                      ReadOnly: false
                      SourceVolume: ipfs
                  PortMappings:
                    - ContainerPort: 5001
                      Protocol: tcp
                      Name: target
            Volumes:
                - Name: ipfs
                  EFSVolumeConfiguration:
                    FilesystemId: !GetAtt EnvControllerAction.ManagedFileSystemID
                    RootDirectory: "/"
                    TransitEncryption: ENABLED
                    AuthorizationConfig:
                        AccessPointId: !Ref AccessPoint
                        IAM: ENABLED
    ExecutionRole:
        Metadata:
            'aws:copilot:description': 'An IAM Role for the Fargate agent to make AWS API calls on your behalf'
        Type: AWS::IAM::Role
        Properties:
            AssumeRolePolicyDocument:
                Version: '2012-10-17'
                Statement:
                    - Effect: Allow
                      Principal:
                        Service: ecs-tasks.amazonaws.com
                      Action: 'sts:AssumeRole'
            Policies:
                - PolicyName: !Join ['', [!Ref AppName, '-', !Ref EnvName, '-', !Ref WorkloadName, SecretsPolicy]]
                  PolicyDocument:
                    Version: '2012-10-17'
                    Statement:
                        - Effect: 'Allow'
                          Action:
                            - 'ssm:GetParameters'
                          Resource:
                            - !Sub 'arn:${AWS::Partition}:ssm:${AWS::Region}:${AWS::AccountId}:parameter/*'
                          Condition:
                            StringEquals:
                                'ssm:ResourceTag/copilot-application': !Sub '${AppName}'
                                'ssm:ResourceTag/copilot-environment': !Sub '${EnvName}'
                        - Effect: 'Allow'
                          Action:
                            - 'secretsmanager:GetSecretValue'
                          Resource:
                            - !Sub 'arn:${AWS::Partition}:secretsmanager:${AWS::Region}:${AWS::AccountId}:secret:*'
                          Condition:
                            StringEquals:
                                'secretsmanager:ResourceTag/copilot-application': !Sub '${AppName}'
                                'secretsmanager:ResourceTag/copilot-environment': !Sub '${EnvName}'
                        - Effect: 'Allow'
                          Action:
                            - 'kms:Decrypt'
                          Resource:
                            - !Sub 'arn:${AWS::Partition}:kms:${AWS::Region}:${AWS::AccountId}:key/*'
                - !If
                  # Optional IAM permission required by ECS task def env file
                  # https://docs.aws.amazon.com/AmazonECS/latest/developerguide/taskdef-envfiles.html#taskdef-envfiles-iam
                  # Example EnvFileARN: arn:aws:s3:::stackset-demo-infrastruc-pipelinebuiltartifactbuc-11dj7ctf52wyf/manual/1638391936/env
                  - HasEnvFile
                  - PolicyName: !Join ['', [!Ref AppName, '-', !Ref EnvName, '-', !Ref WorkloadName, GetEnvFilePolicy]]
                    PolicyDocument:
                        Version: '2012-10-17'
                        Statement:
                            - Effect: 'Allow'
                              Action:
                                - 's3:GetObject'
                              Resource:
                                - !Ref EnvFileARN
                            - Effect: 'Allow'
                              Action:
                                - 's3:GetBucketLocation'
                              Resource:
                                - !Join
                                  - ''
                                  - - 'arn:'
                                    - !Ref AWS::Partition
                                    - ':s3:::'
                                    - !Select [0, !Split ['/', !Select [5, !Split [':', !Ref EnvFileARN]]]]
                  - !Ref AWS::NoValue
            ManagedPolicyArns:
                - !Sub 'arn:${AWS::Partition}:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy'
    TaskRole:
        Metadata:
            'aws:copilot:description': 'An IAM role to control permissions for the containers in your tasks'
        Type: AWS::IAM::Role
        Properties:
            AssumeRolePolicyDocument:
                Version: '2012-10-17'
                Statement:
                    - Effect: Allow
                      Principal:
                        Service: ecs-tasks.amazonaws.com
                      Action: 'sts:AssumeRole'
            Policies:
                - PolicyName: 'DenyIAMExceptTaggedRoles'
                  PolicyDocument:
                    Version: '2012-10-17'
                    Statement:
                        - Effect: 'Deny'
                          Action: 'iam:*'
                          Resource: '*'
                        - Effect: 'Allow'
                          Action: 'sts:AssumeRole'
                          Resource:
                            - !Sub 'arn:${AWS::Partition}:iam::${AWS::AccountId}:role/*'
                          Condition:
                            StringEquals:
                                'iam:ResourceTag/copilot-application': !Sub '${AppName}'
                                'iam:ResourceTag/copilot-environment': !Sub '${EnvName}'
                - PolicyName: 'ExecuteCommand'
                  PolicyDocument:
                    Version: '2012-10-17'
                    Statement:
                        - Effect: 'Allow'
                          Action: ["ssmmessages:CreateControlChannel", "ssmmessages:OpenControlChannel", "ssmmessages:CreateDataChannel", "ssmmessages:OpenDataChannel"]
                          Resource: "*"
                        - Effect: 'Allow'
                          Action: ["logs:CreateLogStream", "logs:DescribeLogGroups", "logs:DescribeLogStreams", "logs:PutLogEvents"]
                          Resource: "*"
                - PolicyName: 'GrantAccessCopilotManagedEFS'
                  PolicyDocument:
                    Version: '2012-10-17'
                    Statement:
                        - Effect: 'Allow'
                          Action:
                            - 'elasticfilesystem:ClientMount'
                            - 'elasticfilesystem:ClientWrite'
                          Condition:
                            StringEquals:
                                'elasticfilesystem:AccessPointArn': !GetAtt AccessPoint.Arn
                          Resource:
                            - Fn::Sub:
                                - 'arn:${partition}:elasticfilesystem:${region}:${account}:file-system/${fsid}'
                                - partition: !Ref AWS::Partition
                                  region: !Ref AWS::Region
                                  account: !Ref AWS::AccountId
                                  fsid: !GetAtt EnvControllerAction.ManagedFileSystemID
    DiscoveryService:
        Metadata:
            'aws:copilot:description': 'Service discovery for your services to communicate within the VPC'
        Type: AWS::ServiceDiscovery::Service
        Properties:
            Description: Discovery Service for the Copilot services
            DnsConfig:
                RoutingPolicy: MULTIVALUE
                DnsRecords:
                    - TTL: 10
                      Type: A
                    - TTL: 10
                      Type: SRV
            HealthCheckCustomConfig:
                FailureThreshold: 1
            Name: !Ref WorkloadName
            NamespaceId:
                Fn::ImportValue: !Sub '${AppName}-${EnvName}-ServiceDiscoveryNamespaceID'
    DynamicDesiredCountAction:
        Metadata:
            'aws:copilot:description': "A custom resource returning the ECS service's running task count"
        Type: Custom::DynamicDesiredCountFunction
        Properties:
            ServiceToken: !GetAtt DynamicDesiredCountFunction.Arn
            Cluster:
                Fn::ImportValue: !Sub '${AppName}-${EnvName}-ClusterId'
            App: !Ref AppName
            Env: !Ref EnvName
            Svc: !Ref WorkloadName
            DefaultDesiredCount: !Ref TaskCount
            # We need to force trigger this lambda function on all deployments, so we give it a random ID as input on all event types.
            UpdateID: 84775c11-08cc-47d7-8710-1f097249a360
    DynamicDesiredCountFunction:
        Type: AWS::Lambda::Function
        Properties:
            Code:
                S3Bucket: stackset-ao-infrastructu-pipelinebuiltartifactbuc-i5qx0wi1qwmz
                S3Key: manual/scripts/custom-resources/dynamicdesiredcountfunction/acd1f00a18ceccc32a780fb208be61f3f62274d775f987fd9feec37493d9173c.zip
            Handler: "index.handler"
            Timeout: 600
            MemorySize: 512
            Role: !GetAtt 'DynamicDesiredCountFunctionRole.Arn'
            Runtime: nodejs16.x
    DynamicDesiredCountFunctionRole:
        Metadata:
            'aws:copilot:description': "An IAM Role for describing number of running tasks in your ECS service"
        Type: AWS::IAM::Role
        Properties:
            AssumeRolePolicyDocument:
                Version: '2012-10-17'
                Statement:
                    - Effect: Allow
                      Principal:
                        Service:
                            - lambda.amazonaws.com
                      Action:
                        - sts:AssumeRole
            Path: /
            ManagedPolicyArns:
                - !Sub arn:${AWS::Partition}:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
            Policies:
                - PolicyName: "DelegateDesiredCountAccess"
                  PolicyDocument:
                    Version: '2012-10-17'
                    Statement:
                        - Sid: ECS
                          Effect: Allow
                          Action:
                            - ecs:DescribeServices
                          Resource: "*"
                          Condition:
                            ArnEquals:
                                'ecs:cluster':
                                    Fn::Sub:
                                        - arn:${AWS::Partition}:ecs:${AWS::Region}:${AWS::AccountId}:cluster/${ClusterName}
                                        - ClusterName:
                                            Fn::ImportValue: !Sub '${AppName}-${EnvName}-ClusterId'
                        - Sid: ResourceGroups
                          Effect: Allow
                          Action:
                            - resource-groups:GetResources
                          Resource: "*"
                        - Sid: Tags
                          Effect: Allow
                          Action:
                            - "tag:GetResources"
                          Resource: "*"
    AutoScalingRole:
        Metadata:
            'aws:copilot:description': 'An IAM role for container auto scaling'
        Type: AWS::IAM::Role
        Properties:
            AssumeRolePolicyDocument:
                Version: '2012-10-17'
                Statement:
                    - Effect: Allow
                      Principal:
                        Service: ecs-tasks.amazonaws.com
                      Action: 'sts:AssumeRole'
            ManagedPolicyArns:
                - !Sub 'arn:${AWS::Partition}:iam::aws:policy/service-role/AmazonEC2ContainerServiceAutoscaleRole'
    AutoScalingTarget:
        Metadata:
            'aws:copilot:description': "An autoscaling target to scale your service's desired count"
        Type: AWS::ApplicationAutoScaling::ScalableTarget
        Properties:
            MinCapacity: 1
            MaxCapacity: 4
            ResourceId:
                Fn::Join:
                    - '/'
                    - - 'service'
                      - Fn::ImportValue: !Sub '${AppName}-${EnvName}-ClusterId'
                      - !GetAtt Service.Name
            ScalableDimension: ecs:service:DesiredCount
            ServiceNamespace: ecs
            RoleARN: !GetAtt AutoScalingRole.Arn
    AutoScalingPolicyECSServiceAverageCPUUtilization:
        Type: AWS::ApplicationAutoScaling::ScalingPolicy
        Properties:
            PolicyName: !Join ['-', [!Ref WorkloadName, ECSServiceAverageCPUUtilization, ScalingPolicy]]
            PolicyType: TargetTrackingScaling
            ScalingTargetId: !Ref AutoScalingTarget
            TargetTrackingScalingPolicyConfiguration:
                PredefinedMetricSpecification:
                    PredefinedMetricType: ECSServiceAverageCPUUtilization
                ScaleInCooldown: 30
                ScaleOutCooldown: 60
                TargetValue: 70
    AutoScalingPolicyECSServiceAverageMemoryUtilization:
        Type: AWS::ApplicationAutoScaling::ScalingPolicy
        Properties:
            PolicyName: !Join ['-', [!Ref WorkloadName, ECSServiceAverageMemoryUtilization, ScalingPolicy]]
            PolicyType: TargetTrackingScaling
            ScalingTargetId: !Ref AutoScalingTarget
            TargetTrackingScalingPolicyConfiguration:
                PredefinedMetricSpecification:
                    PredefinedMetricType: ECSServiceAverageMemoryUtilization
                ScaleInCooldown: 80
                ScaleOutCooldown: 160
                TargetValue: 80
    AutoScalingPolicyALBSumRequestCountPerTarget:
        Type: AWS::ApplicationAutoScaling::ScalingPolicy
        Properties:
            PolicyName: !Join ['-', [!Ref WorkloadName, ALBSumRequestCountPerTarget, ScalingPolicy]]
            PolicyType: TargetTrackingScaling
            ScalingTargetId: !Ref AutoScalingTarget
            TargetTrackingScalingPolicyConfiguration:
                CustomizedMetricSpecification:
                    Dimensions:
                        - Name: LoadBalancer
                          Value: !GetAtt EnvControllerAction.PublicLoadBalancerFullName
                        - Name: TargetGroup
                          Value: !GetAtt TargetGroup.TargetGroupFullName
                    MetricName: RequestCountPerTarget
                    Namespace: AWS/ApplicationELB
                    Statistic: Sum
                ScaleInCooldown: 30
                ScaleOutCooldown: 60
                TargetValue: 10000
    AutoScalingPolicyALBAverageResponseTime:
        Type: AWS::ApplicationAutoScaling::ScalingPolicy
        Properties:
            PolicyName: !Join ['-', [!Ref WorkloadName, ALBAverageResponseTime, ScalingPolicy]]
            PolicyType: TargetTrackingScaling
            ScalingTargetId: !Ref AutoScalingTarget
            TargetTrackingScalingPolicyConfiguration:
                CustomizedMetricSpecification:
                    Dimensions:
                        - Name: LoadBalancer
                          Value: !GetAtt EnvControllerAction.PublicLoadBalancerFullName
                        - Name: TargetGroup
                          Value: !GetAtt TargetGroup.TargetGroupFullName
                    MetricName: TargetResponseTime
                    Namespace: AWS/ApplicationELB
                    Statistic: Average
                ScaleInCooldown: 30
                ScaleOutCooldown: 60
                TargetValue: 2
    EnvControllerAction:
        Metadata:
            'aws:copilot:description': "Update your environment's shared resources"
        Type: Custom::EnvControllerFunction
        Properties:
            ServiceToken: !GetAtt EnvControllerFunction.Arn
            Workload: !Ref WorkloadName
            EnvStack: !Sub '${AppName}-${EnvName}'
            Parameters: [ALBWorkloads, Aliases, EFSWorkloads]
            EnvVersion: v1.13.0
    EnvControllerFunction:
        Type: AWS::Lambda::Function
        Properties:
            Code:
                S3Bucket: stackset-ao-infrastructu-pipelinebuiltartifactbuc-i5qx0wi1qwmz
                S3Key: manual/scripts/custom-resources/envcontrollerfunction/3ffcf03598029891816b7ce2d1ff14fdd8079af4406a0cfeff1d4aa0109dcd7d.zip
            Handler: "index.handler"
            Timeout: 900
            MemorySize: 512
            Role: !GetAtt 'EnvControllerRole.Arn'
            Runtime: nodejs16.x
    EnvControllerRole:
        Metadata:
            'aws:copilot:description': "An IAM role to update your environment stack"
        Type: AWS::IAM::Role
        Properties:
            AssumeRolePolicyDocument:
                Version: '2012-10-17'
                Statement:
                    - Effect: Allow
                      Principal:
                        Service:
                            - lambda.amazonaws.com
                      Action:
                        - sts:AssumeRole
            Path: /
            Policies:
                - PolicyName: "EnvControllerStackUpdate"
                  PolicyDocument:
                    Version: '2012-10-17'
                    Statement:
                        - Effect: Allow
                          Action:
                            - cloudformation:DescribeStacks
                            - cloudformation:UpdateStack
                          Resource: !Sub 'arn:${AWS::Partition}:cloudformation:${AWS::Region}:${AWS::AccountId}:stack/${AppName}-${EnvName}/*'
                          Condition:
                            StringEquals:
                                'cloudformation:ResourceTag/copilot-application': !Sub '${AppName}'
                                'cloudformation:ResourceTag/copilot-environment': !Sub '${EnvName}'
                - PolicyName: "EnvControllerRolePass"
                  PolicyDocument:
                    Version: '2012-10-17'
                    Statement:
                        - Effect: Allow
                          Action:
                            - iam:PassRole
                          Resource: !Sub 'arn:${AWS::Partition}:iam::${AWS::AccountId}:role/${AppName}-${EnvName}-CFNExecutionRole'
                          Condition:
                            StringEquals:
                                'iam:ResourceTag/copilot-application': !Sub '${AppName}'
                                'iam:ResourceTag/copilot-environment': !Sub '${EnvName}'
            ManagedPolicyArns:
                - !Sub arn:${AWS::Partition}:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
    Service:
        Metadata:
            'aws:copilot:description': 'An ECS service to run and maintain your tasks in the environment cluster'
        Type: AWS::ECS::Service
        DependsOn:
            - HTTPListenerRuleWithDomain
            - HTTPSListenerRule
        Properties:
            PlatformVersion: LATEST
            Cluster:
                Fn::ImportValue: !Sub '${AppName}-${EnvName}-ClusterId'
            TaskDefinition: !Ref TaskDefinition
            DesiredCount: !GetAtt DynamicDesiredCountAction.DesiredCount
            DeploymentConfiguration:
                DeploymentCircuitBreaker:
                    Enable: true
                    Rollback: true
                MinimumHealthyPercent: 100
                MaximumPercent: 200
                Alarms: !If
                    - IsGovCloud
                    - !Ref AWS::NoValue
                    - Enable: false
                      AlarmNames: []
                      Rollback: true
            PropagateTags: SERVICE
            EnableExecuteCommand: true
            LaunchType: FARGATE
            ServiceConnectConfiguration:
                Enabled: True
                Namespace: dev.ao.local
                LogConfiguration:
                    LogDriver: awslogs
                    Options:
                        awslogs-region: !Ref AWS::Region
                        awslogs-group: !Ref LogGroup
                        awslogs-stream-prefix: copilot
                Services:
                    - PortName: target
                      # Avoid using the same service with Service Discovery in a namespace.
                      DiscoveryName: !Join ["-", [!Ref WorkloadName, "sc"]]
                      ClientAliases:
                        - Port: !Ref TargetPort
                          DnsName: !Ref WorkloadName
            NetworkConfiguration:
                AwsvpcConfiguration:
                    AssignPublicIp: ENABLED
                    Subnets:
                        Fn::Split:
                            - ','
                            - Fn::ImportValue: !Sub '${AppName}-${EnvName}-PublicSubnets'
                    SecurityGroups:
                        - Fn::ImportValue: !Sub '${AppName}-${EnvName}-EnvironmentSecurityGroup'
            # This may need to be adjusted if the container takes a while to start up
            HealthCheckGracePeriodSeconds: 60
            LoadBalancers:
                - ContainerName: ipfs
                  ContainerPort: 5001
                  TargetGroupArn: !Ref TargetGroup
            ServiceRegistries:
                - RegistryArn: !GetAtt DiscoveryService.Arn
                  Port: !Ref TargetPort
    TargetGroup:
        Metadata:
            'aws:copilot:description': "A target group to connect the load balancer to your service on port 5001"
        Type: AWS::ElasticLoadBalancingV2::TargetGroup
        Properties:
            HealthCheckPath: / # Default is '/'.
            HealthCheckPort: 5001 # Default is 'traffic-port'.
            Matcher:
                HttpCode: 200-399
            Port: 5001
            Protocol: HTTP
            TargetGroupAttributes:
                - Key: deregistration_delay.timeout_seconds
                  Value: 60 # ECS Default is 300; Copilot default is 60.
                - Key: stickiness.enabled
                  Value: false
            TargetType: ip
            VpcId:
                Fn::ImportValue: !Sub "${AppName}-${EnvName}-VpcId"
    RulePriorityFunction:
        Type: AWS::Lambda::Function
        Properties:
            Code:
                S3Bucket: stackset-ao-infrastructu-pipelinebuiltartifactbuc-i5qx0wi1qwmz
                S3Key: manual/scripts/custom-resources/rulepriorityfunction/ac6830d3d4de8167bed1ce48eaf073ccbffe41076a1f88ea5c09b7b0ad71cb14.zip
            Handler: "index.nextAvailableRulePriorityHandler"
            Timeout: 600
            MemorySize: 512
            Role: !GetAtt "RulePriorityFunctionRole.Arn"
            Runtime: nodejs16.x
    RulePriorityFunctionRole:
        Metadata:
            'aws:copilot:description': "An IAM Role to describe load balancer rules for assigning a priority"
        Type: AWS::IAM::Role
        Properties:
            AssumeRolePolicyDocument:
                Version: '2012-10-17'
                Statement:
                    - Effect: Allow
                      Principal:
                        Service:
                            - lambda.amazonaws.com
                      Action:
                        - sts:AssumeRole
            Path: /
            ManagedPolicyArns:
                - !Sub arn:${AWS::Partition}:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
            Policies:
                - PolicyName: "RulePriorityGeneratorAccess"
                  PolicyDocument:
                    Version: '2012-10-17'
                    Statement:
                        - Effect: Allow
                          Action:
                            - elasticloadbalancing:DescribeRules
                          Resource: "*"
    LoadBalancerDNSAlias:
        Metadata:
            'aws:copilot:description': 'The default alias record for the application load balancer'
        Type: AWS::Route53::RecordSetGroup
        Properties:
            HostedZoneId:
                Fn::ImportValue: !Sub "${AppName}-${EnvName}-HostedZone"
            Comment: !Sub "LoadBalancer alias for service ${WorkloadName}"
            RecordSets:
                - Name: !Join
                    - '.'
                    - - !Ref WorkloadName
                      - Fn::ImportValue: !Sub "${AppName}-${EnvName}-SubDomain"
                      - ""
                  Type: A
                  AliasTarget:
                    HostedZoneId: !GetAtt EnvControllerAction.PublicLoadBalancerHostedZone
                    DNSName: !GetAtt EnvControllerAction.PublicLoadBalancerDNSName
    HTTPSRulePriorityAction:
        Metadata:
            'aws:copilot:description': 'A custom resource assigning priority for HTTPS listener rules'
        Type: Custom::RulePriorityFunction
        Properties:
            ServiceToken: !GetAtt RulePriorityFunction.Arn
            RulePath: ["/"]
            ListenerArn: !GetAtt EnvControllerAction.HTTPSListenerArn
    HTTPRuleWithDomainPriorityAction:
        Metadata:
            'aws:copilot:description': 'A custom resource assigning priority for HTTP listener rules'
        Type: Custom::RulePriorityFunction
        Properties:
            ServiceToken: !GetAtt RulePriorityFunction.Arn
            RulePath: ["/"]
            ListenerArn: !GetAtt EnvControllerAction.HTTPListenerArn
    HTTPListenerRuleWithDomain:
        Metadata:
            'aws:copilot:description': 'An HTTP listener rule for path `/` that redirects HTTP to HTTPS'
        Type: AWS::ElasticLoadBalancingV2::ListenerRule
        Properties:
            Actions:
                - Type: redirect
                  RedirectConfig:
                    Protocol: HTTPS
                    Port: 443
                    Host: "#{host}"
                    Path: "/#{path}"
                    Query: "#{query}"
                    StatusCode: HTTP_301
            Conditions:
                - Field: 'host-header'
                  HostHeaderConfig:
                    Values:
                        - Fn::Join:
                            - '.'
                            - - !Ref WorkloadName
                              - Fn::ImportValue: !Sub "${AppName}-${EnvName}-SubDomain"
                - Field: 'path-pattern'
                  PathPatternConfig:
                    Values:
                        - /*
            ListenerArn: !GetAtt EnvControllerAction.HTTPListenerArn
            Priority: !GetAtt HTTPRuleWithDomainPriorityAction.Priority
    HTTPSListenerRule:
        Metadata:
            'aws:copilot:description': 'An HTTPS listener rule for path `/` that forwards HTTPS traffic to your tasks'
        Type: AWS::ElasticLoadBalancingV2::ListenerRule
        Properties:
            Actions:
                - TargetGroupArn: !Ref TargetGroup
                  Type: forward
            Conditions:
                - Field: 'host-header'
                  HostHeaderConfig:
                    Values:
                        - Fn::Join:
                            - '.'
                            - - !Ref WorkloadName
                              - Fn::ImportValue: !Sub "${AppName}-${EnvName}-SubDomain"
                - Field: 'path-pattern'
                  PathPatternConfig:
                    Values:
                        - /*
            ListenerArn: !GetAtt EnvControllerAction.HTTPSListenerArn
            Priority: !GetAtt HTTPSRulePriorityAction.Priority
    AccessPoint:
        Metadata:
            'aws:copilot:description': 'An EFS access point to handle POSIX permissions'
        Type: AWS::EFS::AccessPoint
        Properties:
            ClientToken: !Sub ${AppName}-${EnvName}-${WorkloadName}
            FileSystemId: !GetAtt EnvControllerAction.ManagedFileSystemID
            PosixUser:
                Uid: 0
                Gid: 0
            RootDirectory:
                Path: !Sub '/ipfs'
                CreationInfo:
                    OwnerUid: 0
                    OwnerGid: 0
                    Permissions: '0755'
    AddonsStack:
        Metadata:
            'aws:copilot:description': 'An Addons CloudFormation Stack for your additional AWS resources'
        Type: AWS::CloudFormation::Stack
        DependsOn: EnvControllerAction
        Condition: HasAddons
        Properties:
            Parameters:
                App: !Ref AppName
                Env: !Ref EnvName
                Name: !Ref WorkloadName
            TemplateURL: !Ref AddonsTemplateURL
Outputs:
    DiscoveryServiceARN:
        Description: ARN of the Discovery Service.
        Value: !GetAtt DiscoveryService.Arn
        Export:
            Name: !Sub ${AWS::StackName}-DiscoveryServiceARN
huanjani commented 1 year ago

What does the failure look like/what error message are you getting?

songhobby commented 1 year ago

What does the failure look like/what error message are you getting?

InternalError: failed to create container model: failed to normalize image reference "docker/ipfs/Dockerfile". Launch a new task to retry. Is there anything wrong with the cloudformation.yml?

huanjani commented 1 year ago

Aha. I assume docker/ipfs/Dockerfile is the path to the Dockerfile in your local file system? In your manifest, use image.build (like you had when you posted originally) instead of image.location. See https://aws.github.io/copilot-cli/docs/manifest/lb-web-service/.

songhobby commented 1 year ago

Aha. I assume docker/ipfs/Dockerfile is the path to the Dockerfile in your local file system? In your manifest, use image.build (like you had when you posted originally) instead of image.location. See https://aws.github.io/copilot-cli/docs/manifest/lb-web-service/.

Sorry about the error. After I fixed the syntax error, the deployment gives the old error again chown: /data/ipfs: Operation not permitted Is the override failing? Here is the full manifest.yml 

# Read the full specification for the "Load Balanced Web Service" type at:
#  https://aws.github.io/copilot-cli/docs/manifest/lb-web-service/

# Your service name will be used in naming your resources like log groups, ECS services, etc.
name: ipfs
type: Load Balanced Web Service

# Distribute traffic to your service.
http:
  # Requests to this path will be forwarded to your service.
  # To match all requests you can use the "/" path.
  path: "/"
  # You can specify a custom health check path. The default is "/".
  # healthcheck: '/'
  healthcheck:
    path: "/"
    port: 5001
    success_codes: "200-399"

# Configuration for your containers and service.
image:
  location: ipfs/go-ipfs:v0.10.0
  # Port exposed through your container to route traffic to it.
  port: 5001

cpu: 1024 # Number of CPU units for the task.
memory: 2048 # Amount of memory in MiB used by the task.
# count: 1 # Number of tasks that should be running in your service.
count:
  range: 1-4
  cooldown:
    in: 30s
    out: 60s
  cpu_percentage: 70
  memory_percentage:
    value: 80
    cooldown:
      in: 80s
      out: 160s
  requests: 10000
  response_time: 2s
exec: true # Enable running commands in your container.
network:
  connect: true # Enable Service Connect for intra-environment traffic between services.

# storage:
# readonly_fs: true       # Limit to read-only access to mounted root filesystems.
storage:
  volumes:
    ipfs:
      efs:
        uid: 123456
        gid: 123456
      path: /data/ipfs
      read_only: false
# Optional fields for more advanced use-cases.
#
#variables:                    # Pass environment variables as key value pairs.
#  LOG_LEVEL: info

#secrets:                      # Pass secrets from AWS Systems Manager (SSM) Parameter Store.
#  GITHUB_TOKEN: GITHUB_TOKEN  # The key is the name of the environment variable, the value is the name of the SSM parameter.

# You can override any of the values defined above by environment.
#environments:
#  test:
#    count: 2               # Number of tasks to run for the "test" environment.
#    deployment:            # The deployment strategy for the "test" environment.
#       rolling: 'recreate' # Stops existing tasks before new ones are started for faster deployments.
bvtujo commented 1 year ago

Hey @songhobby, sorry this is still giving you trouble.

I think I've figured out what's going on.

Copilot tries to simplify config of EFS on customers' behalf, so we elided some fields in the access point config that weren't necessary to the majority of use cases. This turned out to be a leaky abstraction, and you're getting bit by the consequences.

I see that you're patching creation info, but this will only work if the directory doesn't exist already in the filesystem. When you delete a service from the environment, you delete its access point CFN resource. Importantly, though, that doesn't actually delete any data in the filesystem (like, say, the directory /ipfs). To get around this, you'll have to tear down the service, specify a new directory or delete the old one, and recreate the service. More details follow.

When an access point is created, the full CFN resource looks like this:

Type: AWS::EFS::AccessPoint
Properties: 
  AccessPointTags: [] # List of k,v tag pairs
  ClientToken: String # Elided since there is no possibility of race conditions on this creation call
  FileSystemId: String # Populated from the environment EFS file system
  PosixUser: 
    Uid: String # From volume.efs.uid
    Gid: String # from volume.efs.gid
    SecondaryGids: [ String ] # elided from the manifest
  RootDirectory: 
    CreationInfo:
      OwnerUid: String # Same as `Uid` above
      OwnerGid: String # Same as `Gid` above
      Permissions: String # Not changeable through copilot, defaults to `0755` (Owner RWX, Group/others RX)
    Path: String # Filled in as `/${WORKLOAD_NAME}`

I think what you may have to do is the following procedure.

  1. Delete this service from the environment with copilot svc delete -e ${ENV_NAME}. This is necessary to trigger deletion of the access point resource so we can recreate it.
  2. Yamlpatch RootDirectory.Path to use a non-existent directory (ipfs-2 or similar) so that the new CreationInfo will actually be applied. If you don't want to create a new directory, you'll have to manually delete the old one. See footnote [^1] for details.
  3. Amend your yamlpatch file to override CreationInfo.OwnerUid, CreationInfo.OwnerGid and CreationInfo.Permissions if desired.
  4. Deploy the new patched service.

Your patch file will look something like this: 

- op: replace
  path: /Resources/AccessPoint/Properties/PosixUser/Uid
  value: 0
- op: replace
  path: /Resources/AccessPoint/Properties/PosixUser/Gid
  value: 0
- op: replace
  path: /Resources/AccessPoint/Properties/RootDirectory/CreationInfo/OwnerUid
  value: 0
- op: replace
  path: /Resources/AccessPoint/Properties/RootDirectory/CreationInfo/OwnerGid
  value: 0
- op: replace
  path: /Resources/AccessPoint/Properties/RootDirectory/Path
  value: ipfs-2

This will create a new directory at /ipfs-2 which your service will transparently mount at /data/ipfs and you can proceed as usual.

Efs Deletion[^2]

Sample EFS deletion manifest ```yaml name: deleter type: Backend Service image: location: amazon/amazon-ecs-sample cpu: 256 memory: 512 count: 1 exec: true storage: volumes: efs: path: /data/efs read_only: false efs: id: ${EFS_FILESYSTEM_ID} ``` You can start a new session with `copilot svc exec` once this manifest is deployed, then a simple `rm -rf /data/efs/ipfs` will remove your existing access point directory. You can then redeploy the patched `ipfs` service.

[^1]: To delete a directory from efs is a little more involved. You'll need a temporary service manifest which mounts the copilot-created filesystem as an external FS. You can get the ID of the filesystem with copilot env show --resources --json | jq '.resources[] | select(.type == "AWS::EFS::FileSystem") | .physicalId'. The sample manifest is linked in footnote 2.

[^2]: Sample EFS deletion manifest

songhobby commented 1 year ago

@bvtujo Appreciate the explanation! I did the following

  1. copilot svc delete
  2. change the copilot/ipfs/overrides/cfn.patches.yml to the above
  3. copilot svc init
  4. copilot svc deploy The deployment still failed with - An EFS access point to handle POSIX permissions [delete complete] [0.0s] Resource handler returned message: "Invalid request" (RequestToken: ec aa0a28-6a90-67d4-2595-b1f3a3752757, HandlerErrorCode: InvalidRequest)
bvtujo commented 1 year ago

This is progress; it looks like I might have given you an invalid YAML patch configuration for the access point resource. Can you run copilot svc package on your ipfs service and paste the access point resource in a comment here? That will help me debug any CFN issues due to overrides.

songhobby commented 1 year ago

This is progress; it looks like I might have given you an invalid YAML patch configuration for the access point resource. Can you run copilot svc package on your ipfs service and paste the access point resource in a comment here? That will help me debug any CFN issues due to overrides.


# Copyright Amazon.com Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: MIT-0
AWSTemplateFormatVersion: 2010-09-09
Description: CloudFormation template that represents a load balanced web service on Amazon ECS using AWS Copilot with YAML patches.
Metadata:
Manifest: |
# The manifest for the "ipfs" service.
# Read the full specification for the "Load Balanced Web Service" type at:
#  https://aws.github.io/copilot-cli/docs/manifest/lb-web-service/
    # Your service name will be used in naming your resources like log groups, ECS services, etc.
    name: ipfs
    type: Load Balanced Web Service

    # Distribute traffic to your service.
    http:
      # Requests to this path will be forwarded to your service.
      # To match all requests you can use the "/" path.
      path: "/"
      # You can specify a custom health check path. The default is "/".
      # healthcheck: '/'
      healthcheck:
        path: "/"
        port: 5001
        success_codes: "200-399"

    # Configuration for your containers and service.
    image:
      location: ipfs/go-ipfs:v0.10.0
      # Port exposed through your container to route traffic to it.
      port: 5001

    cpu: 1024 # Number of CPU units for the task.
    memory: 2048 # Amount of memory in MiB used by the task.
    # count: 1 # Number of tasks that should be running in your service.
    count:
      range: 1-4
      cooldown:
        in: 30s
        out: 60s
      cpu_percentage: 70
      memory_percentage:
        value: 80
        cooldown:
          in: 80s
          out: 160s
      requests: 10000
      response_time: 2s
    exec: true # Enable running commands in your container.
    network:
      connect: true # Enable Service Connect for intra-environment traffic between services.

    # storage:
    # readonly_fs: true       # Limit to read-only access to mounted root filesystems.
    storage:
      volumes:
        ipfs:
          efs:
            uid: 123456
            gid: 123456
          path: /data/ipfs
          read_only: false
    # Optional fields for more advanced use-cases.
    #
    #variables:                    # Pass environment variables as key value pairs.
    #  LOG_LEVEL: info

    #secrets:                      # Pass secrets from AWS Systems Manager (SSM) Parameter Store.
    #  GITHUB_TOKEN: GITHUB_TOKEN  # The key is the name of the environment variable, the value is the name of the SSM parameter.

    # You can override any of the values defined above by environment.
    #environments:
    #  test:
    #    count: 2               # Number of tasks to run for the "test" environment.
    #    deployment:            # The deployment strategy for the "test" environment.
    #       rolling: 'recreate' # Stops existing tasks before new ones are started for faster deployments.

Parameters: AppName: Type: String EnvName: Type: String WorkloadName: Type: String ContainerImage: Type: String ContainerPort: Type: Number TaskCPU: Type: String TaskMemory: Type: String TaskCount: Type: Number DNSDelegated: Type: String AllowedValues: [true, false] LogRetention: Type: Number AddonsTemplateURL: Description: 'URL of the addons nested stack template within the S3 bucket.' Type: String Default: "" EnvFileARN: Description: 'URL of the environment file.' Type: String Default: "" TargetContainer: Type: String TargetPort: Type: Number HTTPSEnabled: Type: String AllowedValues: [true, false] RulePath: Type: String Conditions: IsGovCloud: !Equals [!Ref "AWS::Partition", "aws-us-gov"] HasAssociatedDomain: !Equals [!Ref DNSDelegated, true] HasAddons: !Not [!Equals [!Ref AddonsTemplateURL, ""]] HasEnvFile: !Not [!Equals [!Ref EnvFileARN, ""]] Resources: # If a bucket URL is specified, that means the template exists. LogGroup: Metadata: 'aws:copilot:description': 'A CloudWatch log group to hold your service logs' Type: AWS::Logs::LogGroup Properties: LogGroupName: !Join ['', [/copilot/, !Ref AppName, '-', !Ref EnvName, '-', !Ref WorkloadName]] RetentionInDays: !Ref LogRetention TaskDefinition: Metadata: 'aws:copilot:description': 'An ECS task definition to group your containers and run them on ECS' Type: AWS::ECS::TaskDefinition DependsOn: LogGroup Properties: Family: !Join ['', [!Ref AppName, '-', !Ref EnvName, '-', !Ref WorkloadName]] NetworkMode: awsvpc RequiresCompatibilities:

bvtujo commented 1 year ago

Hmm. I can see one or two possible problems here.

First, the root directory in the access point properties may need a leading / character. I think that's implied by the regex in the access point definition.

Second, the CreationToken field may need to be removed if that's mentioned in the CFN error message.

If those don't fix it, the "last resort" option would be to entirely remove the access point altogether and mount the filesystem at the root using the container's uid and gid. You might have to update your Dockerfile to specify a different uid or gid but this is an option if all else fails.

To do that you'd update your overrides file to remove the Resources.AccessPoint path. Then you'd remove the Resources.TaskDefinition.Properties.Volumes.0.EfsVolumeConfiguration.AuthorizationConfig and Resources.TaskRole.Properties.Policies.2.Condition in order to remove the last reference to the Access Point from the template.

github-actions[bot] commented 1 year ago

This issue is stale because it has been open 60 days with no response activity, and is tagged with pending/question. Remove the stale label, add a comment, or this will be closed in 14 days.

github-actions[bot] commented 1 year ago

This issue is closed due to inactivity. Feel free to reopen the issue if you have any follow-ups!