CI Tests Failing Due to Docker Hub Rate Limiting on Image Pulls

Recently we've had issues with with CI throwing an error on tests that depend on docker to pull images of services the test depends on. The problem is most likely due to the excessive amount of pulling services for each test environment. For example e2e tests will need to pull at minimum 3 images (kafka, elasticsearch, and zookeeper). This is multiplied by the matrix that will test different versions of node and then multiplied again with different versions of elasticsearch.

Error:

toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit

Link: https://github.com/terascope/teraslice/actions/runs/9784466833/job/27015404404

Quoted from docker itself: https://www.docker.com/increase-rate-limits/

The rate limits of 100 container image requests per six hours for anonymous usage, and 200 container image requests per six hours for free Docker accounts are now in effect. Image requests exceeding these limits will be denied until the six hour window elapses.

Will look into this more and add potential solutions soon.

terascope / teraslice

CI Tests Failing Due to Docker Hub Rate Limiting on Image Pulls #3676