search

nienbo / cache-buildkite-plugin

Tarball, Rsync & S3 Cache Kit for Buildkite. Supports Linux, macOS and Windows
https://buildkite.com/plugins
MIT License
67 stars 39 forks source link

Failed S3 downloads not handled #64

Closed bradleyayers closed 11 months ago

bradleyayers commented 12 months ago

We're using the S3 backend and experiencing an issue where corrupted .tar.gz files are being downloaded and extracted but tar dies because it can't decompress the file.

image

I think it might be happening if the aws s3 cp command fails part way through (our files are ~400MB so perhaps the connection sometimes breaks), and then the script proceeds anyway to copy it into the local cache location:

https://github.com/nienbo/cache-buildkite-plugin/blob/9df04eba98325d751e108b5f1e9ec59d21205020/lib/backends/s3.bash#L115

Do we need something like a || exit 1?

gencer commented 12 months ago

Perhaps, But I wonder how this works on different shells. I've seen some weird behaviors on different shells using boolean checks with ||

I also commented on your PR.

bradleyayers commented 11 months ago

I've raised #65 to resolve this.

nuzayets commented 11 months ago

Thank you for patching it! Right as we've started to hit this issue, too.

Could we get a patch version for this soon, pretty please? 🙏

gencer commented 11 months ago

Released as v2.4.15 🚀