Adds a job at the beginning of the build workflow which updates caches containing the different submodules required to build the different variants.
This job takes about 3 minutes to run the first time, and after that it takes about 4 seconds to check that the cache is still up to date. This could also be moved to a different trigger which runs less often.
This greatly reduces the amount of resources used to clone submodules (just for the build variants for now). The clones that do happen are shallow (except for uclibc-ng) which also shrinks the git overhead by at least 1GB.
This action only caches the .git/modules directory as the rest of the working directory can be rebuilt from those without needing to download anything else. This allows us to choose which submodules we want to manifest into our build environment. The directory sizes for GCC and LLVM are 1.1GB and 1.9GB respectively, so being able to pick which we want in our environment saves a lot of disk space.
An additional note on github caches. They are scoped to branches, but any branch can restore a cache that was saved by master. Repos are limited to 10GB of total cache storage, so restricting cache generation to master may also be a good move to prevent using all of the cache storage capacity.
TShapinsky
commented
3 weeks ago
Adds a job at the beginning of the build workflow which updates caches containing the different submodules required to build the different variants.
This job takes about 3 minutes to run the first time, and after that it takes about 4 seconds to check that the cache is still up to date. This could also be moved to a different trigger which runs less often.
This greatly reduces the amount of resources used to clone submodules (just for the build variants for now). The clones that do happen are shallow (except for uclibc-ng) which also shrinks the git overhead by at least 1GB.
This action only caches the
.git/modulesdirectory as the rest of the working directory can be rebuilt from those without needing to download anything else. This allows us to choose which submodules we want to manifest into our build environment. The directory sizes for GCC and LLVM are 1.1GB and 1.9GB respectively, so being able to pick which we want in our environment saves a lot of disk space.An additional note on github caches. They are scoped to branches, but any branch can restore a cache that was saved by
master. Repos are limited to 10GB of total cache storage, so restricting cache generation tomastermay also be a good move to prevent using all of the cache storage capacity.