quicklisp / quicklisp-controller

The software to build Quicklisp dists.
Other
38 stars 12 forks source link

Make the build process easy to parallelize #4

Open quicklisp opened 11 years ago

quicklisp commented 11 years ago

In d2a68450e7b6acd027412f2364c3be1ac1622a08 @orivej made it possible to map over the source objects in parallel. Unfortunately, the build process of ensure-what-wins-you-can assumes it can use fixed locations on the filesystem as temporary storage for various build steps, like calculating making a dist tarball, computing the tarball's digest, and possibly other things. These steps clobber each other when run in parallel.

I'd like to make a new backend for ensure-what-wins-you-can that works within a sandboxed subdirectory for each system it tries to build.

The build functionality can be put into a separate binary that takes the system index, system name, and sandbox directory as command-line options, then does what it needs to put cache files, FASLs, temporary files, logs, output metadata, etc into the sandbox.

Then a certain number of those separate binaries can be run at the same time to reduce the amount of time the build takes overall.