eddmann / serverless-fargate

Maintain long-running Fargate ECS tasks within your Serverless project
53 stars 22 forks source link

Possible to add Deployment circuit breaker? #39

Closed QAnders closed 4 months ago

QAnders commented 1 year ago

On the Service, there's a property called Deployment circuit breaker, that when turned on has the feature to rollback on failures and terminate deploy.

This is quite convenient if some hick-up in config or code prevents the deployment and the previous revision is automatically put back.

Would it be possible to use this switch from the Serverless.yml setup?

Thanks for a great plugin!

eddmann commented 1 year ago

Hey, I think that's a great addition! At this time you should be able to configure it using the cloudFormationResource definition which would override the entire DeploymentConfiguration for you to define as you wish.

But I will look at getting this option in on the next release!

zacroker commented 1 year ago

@QAnders you may have already solved this, but you can add it to the extensions block of your resources too so it will be merged with the configuration that gets emitted from the package.

org: xxx
app: xxx
service: xxx
provider:
    xxx:

...

resources:
  - xxx
  - xxx
  - extensions:
    HeliosceleryworkerService:
    Properties:
      DeploymentConfiguration:
        DeploymentCircuitBreaker:
          Enable: true
          Rollback: true
      EnableExecuteCommand: true
...
QAnders commented 1 year ago

Ah, thanks, @zacroker!

No, I hadn't figured it out, but noticed if you deploy, and then set it from AWS Console (or CLI) it'll "stick" and remain for the next deploy so that's what we did...

QAnders commented 1 year ago

Coming back to update this, as we had a bit of an issue getting it done... The name to provide for the extension, is not the ECS service! It's the Cloudformation service name and I'm not exactly sure how this is constructed, but our Fargate service in the Serverless.yml is server-publicapi, and the ECS service name, is thus, the same.

The extension must have the name ServerpublicapiService though, where the case is important, so my guess is that it takes the Fargate service name, removes any dashes and then capitalizes the first letter and appends Service to the end...

You can also see the actual name in the Cloudformation events.

resources:
  extensions:
    ServerpublicapiService:
      Properties:
        DeploymentConfiguration:
          DeploymentCircuitBreaker:
            Enable: true
            Rollback: true
        EnableExecuteCommand: true
  Resources:
    storeMDSQueue:
      Type: AWS::SQS::Queue
      Properties:
  ...
  ...
zacroker commented 1 year ago

I have also since learned how to do it another way (without having to determine the cloudformation logical resource name). As indicated by @eddmann you can use the cloudFormationResource key under the fargate key. To add configuration to the service, use the service key. I missed it when I first looked through the docs, but it is in the README (the following block has been cut/paste from there)

...
  # (optional) additional CloudFormation resource properties you wish to include for all compiled tasks.
  # This provides a means of including properties present in the CloudFormation resource that are not exposed via the plugin DSL.
  cloudFormationResource:
    # https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ecs-taskdefinition.html
    task:
      EphemeralStorage: 5
    # https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ecs-taskdefinition-containerdefinitions.html
    container:
      StopTimeout: 5
    # container definitions you wish to run alongside the primary container
    # https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ecs-taskdefinition-containerdefinitions.html
    additionalContainers:
      - Name: 'additional-container-name'
        Image: 'additional-container-image'
    # https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ecs-service.html
    service:
      EnableECSManagedTags: 'true'
QAnders commented 1 year ago

I have also since learned how to do it another way (without having to determine the cloudformation logical resource name). As indicated by @eddmann you can use the cloudFormationResource key under the fargate key. To add configuration to the service, use the service key. I missed it when I first looked through the docs, but it is in the README (the following block has been cut/paste from there)

Thanks! Ironically though, that's actually what I initially tried but it wasn't working... Here's the code I added some time ago:

      cloudFormationResource:
        container:
          PortMappings:
            - ContainerPort: 8080
        service:
          # DeploymentConfiguration: # NOT WORKING! Would be nice to have though...
          #  DeploymentCircuitBreaker:
          #    Enable: true
          #    Rollback: true
          LoadBalancers:
            ...
            ...

We removed that as it wasn't doing anything... The above comment works though!

eddmann commented 4 months ago

Reading through this all looks to be resolved now? Please reply back if you still have any issues to discuss.