search

eth-cscs / stackinator

https://eth-cscs.github.io/stackinator/
BSD 3-Clause "New" or "Revised" License
19 stars 15 forks source link

Easy pushing of partial builds to the build cache #133

Closed bcumming closed 1 year ago

bcumming commented 1 year ago

Stackinator pushes all built packages to the build cache once all packages in individual spack environment has been successfully built.

If an error occurs towards the end of building an environment, the already built packages are not pushed to the cache. For large environments this can waste a lot of developer time.

Proposed solution: provide a script named something like force-push-cache that will push all packages that have been installed in the build tree in ${build_path}/store to the build cache. Make this opt-in, because fixing the underlying issue might require rebuilding a different set of dependencies, and we want to avoid pushing unnecessary packages to the build cache.

biddisco commented 1 year ago

Thoughts... Option 1 - provide a script that the user runs as part of the post install process (is that feasible/sensible to put it in the post install script if we opt-in?) Option 2 - same as above, but also add a flag in the cache config yaml files that allows the user to just specify that all builds are pushed and then have it done automagically by stackinator

bcumming commented 1 year ago

The post install script is only run if the stack has been built successfully, i.e. when all the targets required to build store.squashfs have been completed successfully. So that approach wouldn't address our specific concern.

I could add a makefile target make cache-push?