Loading cache on AWS CodeBuild runner is way slower than on GitHub runner

[garysassano]

I'm using CodeBuild-hosted GitHub Actions runners and I've noticed something quite interesting when building a dozen of Docker images in parallel: loading the layers from the GitHub Actions Cache is way slower compared to the official GitHub runners hosted on Azure.

Is there any workaround for this? I understand that latency would be lower on the official GitHub runners since they live on the same Azure network as the GitHub Actions Cache, but the impact shouldn't be so big.

Isn't there any Docker cache alternative for AWS users?

[garysassano]

Apparently, you could use ECR cache or S3 cache. I still need to test if they're faster, but I’d expect them to be.

[garysassano]

Result using ECR cache.

[garysassano]

Result using S3 cache.

It consistently outperformed ECR (tested by running the same workflow 10 times consecutively). I believe this is because each layer is stored as a separate blob, allowing BuildKit to selectively download only the necessary layers after retrieving the cached manifest, instead of downloading the entire image manifest as ECR does.