shawnbot
commented
8 years ago One thing that will be really tricky if we decide to do this is parallelizing the Sauce Connect setup. Right now spinning up an HTTP server on localhost and making it available to Sauce Labs via a secure tunnel, and each build gets a unique tunnel ID so that Sauce tests the right code. Multiple containers probably can't use the same tunnel ID, so we'd have to ensure that only one container runs the HTTP server and creates the tunnel, which the parallelized containers would use.
The alternative is an approach that I've also considered for EITI (cc @gemfarmer), in which we skip the Jekyll build and HTTP server on CI altogether and run our CI tests against the public Federalist preview URL for each branch. The only tricky thing about this approach is the timing: Federalist and Circle get the github webhook at the same time, and if Federalist takes longer to build the site than Circle does to spin up a build container, then the tests will run against an older (or non-existent, in the case of the a first branch push) version of the site.
Now that Federalist gives us commit info in the site object, though, we can have our CI build wait for Federalist to finish by requesting a file containing the commit SHA:
---
---
{{ site.commit.commit }}against which we would compare the $CIRCLE_SHA1 environment variable and exit once they match.
Site builds (not test builds) might be slower on Federalist than they are on Circle, but this could reduce a lot of the CI complexity. cc @yozlet
We can apparently parallelize some of our tests on Circle CI. If this is possible (and relatively straightforward), we should do it! At the very least our unit, browser, and a11y test suites could run in parallel and probably reduce the build time by a couple of minutes. This would in theory speed up the rate of development and help people catch bugs more quickly.