search

aws / copilot-cli

The AWS Copilot CLI is a tool for developers to build, release and operate production ready containerized applications on AWS App Runner or Amazon ECS on AWS Fargate.
https://aws.github.io/copilot-cli/
Apache License 2.0
3.52k stars 414 forks source link

Performance of svc package in a build pipeline #5873

Open srbartlett opened 4 months ago

srbartlett commented 4 months ago

Hey all,

Before I start investigating options for how to improve my build speed, I wanted to ask for feedback here.

The problem is my CI takes about 30 minutes to create a various docker images for my pipeline. Yes, improving the Docker build performance is on my list of thing to do. I tackled this in the past

But I am also keen to explore options to parallelize the build. The line I am looking at:

DOCKER_BUILDKIT=1 ./copilot-linux svc package -n $svc -e $env --output-dir './infrastructure' --tag $tag --upload-assets;

Has any one attempted to parallelize this? Assuming I have provisioned an adequate instance, it should be possible right? If so, would forking the process or using native AWS code build feature be recommended?

Any tips or advice would be appreciated.

Thanks!

iamhopaul123 commented 4 months ago

Hello @srbartlett do you want to run multiple svc package in parallel or it's the same svc package with multiple sidecar containers? Because we've already supported building main image with sidecar images in parallel.

srbartlett commented 4 months ago

Hi @iamhopaul123,

Yes, I want to run multiple ⁠svc package commands in parallel for different stages in my pipeline, such as QA, staging, and production. Each stage has a load balancer service and a backend service. I don’t think using sidecar parallelization would help, would it?

iamhopaul123 commented 4 months ago

I don’t think using sidecar parallelization would help, would it?

No, I don't think so in this case. Sorry about that but I think you would need to update the buildspec file (shell script) with this logic on your own and there's no native support in Copilot for this.