microsoft / ga4gh-tes

C# implementation of the GA4GH TES API; provides distributed batch task execution on Microsoft Azure
MIT License
32 stars 26 forks source link

tes-runner should retry Docker image notfound #720

Closed MattMcL4475 closed 2 months ago

MattMcL4475 commented 2 months ago

Problem: It seems that GCR returns "notfound" when rate-limiting, which causes the task to fail: Docker.DotNet.DockerImageNotFoundException: Docker API responded with status code=NotFound, response={"message":"No such image: us.gcr.io/broad-gatk/gatk@sha256:53b4590c3c8d2d5db7a02c07754a69ea134841f6a08a5a16b238bbf2d6ebd250"}

Solution: Retry, but add a note in the comments that this is a workaround for a suspected GCR bug and should be removed later if confirmed