eve tests were being slowed down, and at times completely blocked, due to Docker Hub pull rate limits; see this eve issue. Mostly, that has been fixed, due to a variety of removing unnecessary pulls, ensuring dependencies are part of docker OSS and having the Docker Hub ID used to perform the pulls run under an LFEdge paid account that allows up to 5,000 pulls per day.
eden tests still pull many images, as part of:
starting eden, e.g. redis or s3
running eve, eve images, although this is not an issue for 2 reasons: eve images are part of Docker-sponsored OSS and exempt from rate limits, and eden tests mostly take images from earlier GitHub Actions artifacts, rather than pulling from an OCI registry
running actual tests
The Docker Hub ID used for eden tests, especially those imported into eve as part of the eve CI pipeline, should be running under the paid ID provided by LFEdge. However, these will hit limits eventually.
This issue is to track finding where images are pulled as part of the eden suite, and find ways to avoid hitting rate limits via:
eliminating unnecessary pulls
restricting pulls to Docker sponsored OSS images, e.g. anything published under lfedge/*
sharing pulls among runs
Because eve runs a matrix of different eden tests, if each eden test run pulls "only" 5 images, and the matrix itself runs 5 tests, that is 25 tests for each such PR, not counting any others. Since most of the runs duplicate images, this scales exponentially.
The goal is to get to the point, where each PR needs just a few images pulls other than sponsored (rate limit-exempt) images.
eve tests were being slowed down, and at times completely blocked, due to Docker Hub pull rate limits; see this eve issue. Mostly, that has been fixed, due to a variety of removing unnecessary pulls, ensuring dependencies are part of docker OSS and having the Docker Hub ID used to perform the pulls run under an LFEdge paid account that allows up to 5,000 pulls per day.
eden tests still pull many images, as part of:
The Docker Hub ID used for eden tests, especially those imported into eve as part of the eve CI pipeline, should be running under the paid ID provided by LFEdge. However, these will hit limits eventually.
This issue is to track finding where images are pulled as part of the eden suite, and find ways to avoid hitting rate limits via:
lfedge/*Because eve runs a matrix of different eden tests, if each eden test run pulls "only" 5 images, and the matrix itself runs 5 tests, that is 25 tests for each such PR, not counting any others. Since most of the runs duplicate images, this scales exponentially.
The goal is to get to the point, where each PR needs just a few images pulls other than sponsored (rate limit-exempt) images.