search

compose-x / ecs_composex

Manage, Configure and Deploy your services and AWS services and applications from your docker-compose definitions
https://docs.compose-x.io
Mozilla Public License 2.0
158 stars 16 forks source link

[BUG] Cannot launch service with EC2 launch type #698

Closed fala1991 closed 7 months ago

fala1991 commented 8 months ago

Describe the bug I have set deploy.labels with ecs.compute.platform: EC2 and got log - At least one service defined for EC2. Overriding for all however, the task definition still set LaunchType to FARGATE

ecs-compose-x up -d outputs -n demo-sift -f docker-compose.yaml                                        
/opt/homebrew/lib/python3.11/site-packages/ecs_composex/compose/compose_services/service_image/docker_opts.py:33: UserWarning: Run pip install ecs-composex[ecrscan] to enable this functionality.
  warnings.warn(str(error))
2023-11-09 10:55:51 [    INFO] Validating against input schema /opt/homebrew/lib/python3.11/site-packages/ecs_composex/specs/compose-spec.json
2023-11-09 10:55:51 [    INFO] sift - At least one service defined for EC2. Overriding for all
2023-11-09 10:55:51 [ WARNING] At least one service requires cloudmap or AppMesh is used. Enabling private namespace
2023-11-09 10:55:52 [    INFO] Service families to process ['sift']
Loaded x-vpc vpc vpc /opt/homebrew/lib/python3.11/site-packages/ecs_composex/vpc
Loaded x-cloudmap cloudmap cloudmap /opt/homebrew/lib/python3.11/site-packages/ecs_composex/cloudmap
2023-11-09 10:55:52 [    INFO] Processing x-vpc
2023-11-09 10:55:54 [    INFO] x-vpc - Found VPC - vpc-0142727385489fa4d
2023-11-09 10:55:54 [    INFO] Processing x-cloudmap
2023-11-09 10:55:55 [    INFO] x-vpc - Found VPC - vpc-0142727385489fa4d
2023-11-09 10:55:56 [    INFO] networks.private - mapped x-vpc.AppSubnets to sift
2023-11-09 10:55:56 [    INFO] sift - Hostname set to sift
2023-11-09 10:55:56 [    INFO] sift - Ephemeral storage: 0
2023-11-09 10:55:56 [    INFO] sift - LaunchType set to FARGATE
2023-11-09 10:55:56 [    INFO] sift - TaskDefinition containers: ['sift']
2023-11-09 10:55:58 [    INFO] iam.json uploaded successfully to https://s3.amazonaws.com/ecs-composex-789798371969-us-west-2/2023/11/09/1855/ce15c0/iam.json
2023-11-09 10:55:59 [    INFO] sift.json uploaded successfully to https://s3.amazonaws.com/ecs-composex-789798371969-us-west-2/2023/11/09/1855/ce15c0/sift.json
2023-11-09 10:56:00 [    INFO] sift.params.json uploaded successfully to https://s3.amazonaws.com/ecs-composex-789798371969-us-west-2/2023/11/09/1855/ce15c0/sift.params.json
2023-11-09 10:56:00 [    INFO] sift.config.json uploaded successfully to https://s3.amazonaws.com/ecs-composex-789798371969-us-west-2/2023/11/09/1855/ce15c0/sift.config.json
2023-11-09 10:56:00 [    INFO] cloudmap.json uploaded successfully to https://s3.amazonaws.com/ecs-composex-789798371969-us-west-2/2023/11/09/1855/ce15c0/cloudmap.json
2023-11-09 10:56:00 [    INFO] demo-sift.json uploaded successfully to https://s3.amazonaws.com/ecs-composex-789798371969-us-west-2/2023/11/09/1855/ce15c0/demo-sift.json
2023-11-09 10:56:01 [    INFO] UPDATE_IN_PROGRESS

To Reproduce Steps to reproduce the behavior:

  1. pip install ecs-composex
  2. How you are executing it ? : As a CLI tool, as a library, in lambda? : ecs-compose-x up -d outputs -n demo-sift -f docker-compose.yaml
  3. Link / share your Docker compose file 
    
---
# Base docker-compose file

version: "3.8"

x-vpc: Lookup: VpcId: Tags:

networks: public: x-vpc: PublicSubnets private: x-vpc: AppSubnets

x-cloudmap: subzero: Name: subzero.private Lookup: NamespaceId: ns-dcac5ueihkz5gh5c

services: sift: networks:

x-cluster: Properties: CapacityProviders:


6. See error

**Expected behavior**
this log should say EC2
2023-11-09 10:55:56 [    INFO] sift - LaunchType set to EC2

**Logs**
If applicable, add logs of errors using from ECS ComposeX

**Desktop (please complete the following information):**
 - OS: [e.g. Fedora/Windows/Mac]
 - Python version (3.6,3.7,3.8)
 - If you were doing things from a branch, which one?
 - Commit ID ?
 - Tag/Version?
JohnPreston commented 8 months ago

Thanks for the report @fala1991 I have just last week deployed to EC2, I will dig into why this didn't work.

I noticed

x-cluster:
  Properties:
    CapacityProviders:
      - subzero-inference
      - FRGATE
    ClusterName: subzero-demo

Noticing the typo, on for FARGATE, did the cluster get created properly?

edit1: what's the value of parameter EcsLaunchType on your stack?

JohnPreston commented 8 months ago

Okay, behaviour reproduced with

services:
  test:
    image: nginx
    deploy:
      labels:
        ecs.compute.platform: EC2
      resources:
        reservations:
          cpus: '0.1'
          memory: 64MB
        limits:
          cpus: '0.25'
          memory: 128MB

x-cluster:
  Properties:
    CapacityProviders:
      - subzero-inference
      - FRGATE
    ClusterName: subzero-demo
fala1991 commented 8 months ago

I updated the typo with 

services:
  sift:
    networks:
      - private
    image: 789798371969.dkr.ecr.us-west-2.amazonaws.com/stg-ins-sift:d048178-dirty
    ports:
      - 8080:8080
    labels:
      container_name: sift
    x-network:
      x-cloudmap: subzero
      Ingress:
        ExtSources:
          - IPv4: 10.238.216.0/22
            Name: all
    x-iam:
      Policies:
        - PolicyName: TraefikRecommended
          PolicyDocument: {
            "Version": "2012-10-17",
            "Statement": [
              {
                "Sid": "TraefikECSReadAccess",
                "Effect": "Allow",
                "Action": [
                    "ecs:ListClusters",
                    "ecs:DescribeClusters",
                    "ecs:ListTasks",
                    "ecs:DescribeTasks",
                ],
                "Resource": "*"
              }
            ]
          }
    deploy:
      labels:
        ecs.compute.platform: EC2
      resources:
        reservations:
          cpus: '0.1'
          memory: 64MB
        limits:
          cpus: '0.25'
          memory: 128MB

x-cluster:
  Properties:
    CapacityProviders:
      - subzero-inference
      - FARGATE
    ClusterName: subzero-demo

Still have same issue. the value of EcsLaunchType is still Fargate

JohnPreston commented 8 months ago

Yeah, I didn't expect that to fix it, I just wanted to point it out. I am curious to have the details of the ECS cluster though purely out of interest. Don't have to share if you prefer not to of course

I am pinpointing where the change happens EDIT: Found where it changes

fala1991 commented 8 months ago

is there anyway to force launch type to EC2 except ecs.compute.platform: EC2?

JohnPreston commented 8 months ago

is there anyway to force launch type to EC2 except ecs.compute.platform: EC2?

So Compose-x tries to be smart, and correct the launch type if the cluster didn't support it. Might be trying to be too smart. It will do that based on the settings/providers it finds in the cluster. But you are creating the cluster. Technically a cluster doesn't need to say it uses EC2 as the capacity provider, it would do that if there are ECS instances and you didn't request to use Fargate instead for example.

I found where in the code it does the change, now to find the why. I presume that the provider subzero-inference is your configured ASG?

JohnPreston commented 8 months ago

Okay, found where it changes but shouldn't.

fala1991 commented 8 months ago

yes subzero-inference is my ASG group and I have made a call to create that as capacity provider via 

aws ecs create-capacity-provider --name subzero-inference --auto-scaling-group-provider "autoScalingGroupArn=arn:aws:autoscaling:us-west-2:789798371969:autoScalingGroup:8490889d-2a99-44af-9352-60010bed5357:autoScalingGroupName/subzero-inference"
fala1991 commented 8 months ago

Should I just use existing cluster? will that solve my issue?

JohnPreston commented 8 months ago

Should I just use existing cluster? will that solve my issue?

Most likely yes, or setting x-ecs.CapacityProviders and point to your should do the trick. But I want to fix this anyways. Sorry I had to disappear to go make a fool of myself: https://www.youtube.com/watch?v=1glvMLqX8bg :D

JohnPreston commented 8 months ago

Okay, tested with

services:
  test:
    image: nginx
    deploy:
      labels:
        ecs.compute.platform: FARGATE
      resources:
        reservations:
          cpus: '0.1'
          memory: 64MB
        limits:
          cpus: '0.25'
          memory: 128MB

  test-sidecar:
    image: nginx
    deploy:
      labels:
        ecs.compute.platform: EC2
        ecs.task.family: test
      resources:
        reservations:
          cpus: '0.1'
          memory: 64MB
        limits:
          cpus: '0.25'
          memory: 128MB

x-cluster:
  Properties:
    CapacityProviders:
      - subzero-inference
      - FRGATE
    ClusterName: subzero-demo

and other variances (EXTERNAL for example) and the highest priority wins. Priority goes EXTERNAL> EC2 > FARGATE => Fargate is default, so using EXTERNAL or EC2 is a user choice. EXTERNAL takes priority over EC2

JohnPreston commented 7 months ago

@fala1991 sorry forgot to tag you on my earlier message to notify you. Let me know how this goes?