search

AdlerFarHorizons / NITELite-pipeline

Code to process images and data taken with the HAB NITELite instrument.
GNU General Public License v3.0
1 stars 7 forks source link

docker-nitelite-pipeline routinely fails because AWS is sending too many requests to Docker Hub #19

Open zhafen opened 7 months ago

zhafen commented 7 months ago

Describe the bug Our docker image depends on a miniconda image hosted on Docker Hub, so we download it at the start of building our image (this is in aws/image-buildspec.yaml). Unfortunately, because we share an IP address with a large number of other AWS users then Docker Hub routinely rate limits our request. This results in the job failing.

To Reproduce Run the docker-nitelite-pipeline CodeBuild project with default settings during work hours.

Expected behavior The pull to happen successfully.

Screenshots

image

Example error.

Suggested solution We host a mirror of miniconda on our Adler account, and pull from it.