Question about Continuous Integration / Automated Builds

[MedLexAI]

I apologize in advance if this is not the right forum for questions, I didn't see any reference to a mailing list on your github.

I have a Continuous Integration pipeline in place for automating the build process for our Golang-based Docker images. Basically whenever there is any type of a code commit to the web application framework, new Go binaries are built followed by an updated Dockerfile and with all of those changes pushed to the Amazon private ECR where our Fargate tasks are cancelled and then relaunched with the latest Docker ECR image.

Does your fargate CLI have any support yet for detecting when a Fargate task image has been updated, so that those running tasks can be cancelled and then re-launched (preferably one at a time within a load balancer so as not to disrupt the application)?

Or should I just integrate the fargate CLI into the task reload process once the updated Docker image has been pushed to ECR?

Thanks in advance!

[siddarthsreeni-zz]

I recommend using a simple queue system. (aws sqs) instead.

[jpignata]

No reason to apologize! This is absolutely the right place.

What you're describing is an ECS service. A service will maintain a given number of running tasks, and will deploy new versions of your task definitions across your service. This CLI has support for creating services and a convenience command (service deploy) which will push to ECR, create a new task definition, and update a service to point to that task definition.

In terms of a CI/CD pipeline, the deploy stage for ECS is the above, so folks have used this CLI to drive the deploy stage of their pipeline from within tools and services like Jenkins and Travis CI. There are other approaches to consider. For example, this reference architecture uses AWS' native developer tools (CodeBuild, CodePipeline), the latter of which has a native integration with ECS which creates a new task definition and updates the service.

Hope that helps! Let me know if you need any other details.

[MedLexAI]

This is perfect, thank you. I was able to create a load balanced service today and plan on testing the deploy option in your CLI next.

[MedLexAI]

Alright, testing this deploy option and I'm having some issues again.

From the src directory for my Golang project, I have a Dockerfile created that I am trying to push with the fargate cli deploy option.

$ fargate service deploy testapp
[!] Couldn't describe Amazon ECR repositories
RepositoryNotFoundException: The repository with name 'testapp' does not exist in the registry with id '1234567890'
        status code: 400, request id: 12345678-45678-12345678

testapp is definitely a part of the 1234567890 private ECR repo, and in fact is the current running task in the service that I created earlier this morning.

Any idea what I am doing wrong with this?

[jpignata]

That's interesting. service create creates a repository in your AWS account, but this seems to suggest that repository was either not successfully created or deleted. Can you attempt to run:

aws ecr describe-repositories --repository-name testapp

You should see:

{
    "repositories": [
        {
            "registryId": "1234567890",
            "repositoryName": "testapp",
            "repositoryArn": "arn:aws:ecr:us-east-1:1234567890:repository/blah",
            "createdAt": 1518029931.0,
            "repositoryUri": "1234567890.dkr.ecr.us-east-1.amazonaws.com/testapp"
        }
    ]
}

If you see:

An error occurred (RepositoryNotFoundException) when calling the DescribeRepositories operation: The repository with name 'testapp' does not exist in the registry with id '1234567890'

.. the repository either wasn't created or deleted, oddly. I'd check CloudTrail to see what operations were ran against ECR since your deploy. You can recreate your repo via:

aws ecr create-repository --repository-name testapp

[MedLexAI]

Hmm. Under that repo, it's setup as companyname/testapp.

aws ecr describe-repositories --repository-name testapp

results with "An error occured"...

But aws ecr describe-repositories --repository-name companyname/testapp

returns with the correct repo information.

However,

$ fargate service deploy companyname/testapp
[!] Could not describe ECS service
Could not find companyname/testapp

?

[jpignata]

Did the CLI create companyname/testapp? I've tried to simplify this CLI as much as possible so it hides the ECR business from the user. The happy path is that you create a service, and it creates a repo in the backend using a conventional name (in this case, the name of the service) and uses that. I don't see how you got into a state where your repository is named differently than your service..? Can you provide more details about what you did to get into that state?

[MedLexAI]

I created another repository, but this time just named testapp instead of companyname/testapp.

The repository was created using the ECR GUI interface.

I was then able to tag and push a local Docker image up to ECR, and then launch the image using fargate task run testapp --image etc

aws ecr describe-repositories --repository-name testapp works fine

But same issue when I try to create a service with it:

$ fargate service deploy testapp
[!] Could not describe ECS service
Could not find testapp

[jpignata]

I'm sorry - I'm pretty confused. Did you run fargate service create testapp at any point? That's really how this tool is meant to be used: a service is created and deployed to, and it sets up the necessary infrastructure.