Closed cockroach-teamcity closed 5 years ago
^- this hit all (or at least a lot of) runs that night. Is this just a rare fluke? Were there recent changes that could've caused this? cc @bdarnell There haven't been any recent changes to the jepsen infra that I'm aware of.
lein
is the clojure package manager/runner/kitchen sink, installed by this line. No changes on our side, nor have there been any recent releases of lein
. I think it's probably a rare github/network flake combined with error handling that's not quite right (looks like we need to add the -f
flag to curl to have it report failure as expected).
Oh, the problem is that if we hit the apt-get failure (which is recently skipped by #37430), we still write the jepsen_initialized
file (from a defer
), which causes subsequent test runs to proceed on the incompletely-initialized cluster (the apt step comes before the lein installation). I have no idea why that's a defer
; it looks like it should only be written on successful initialization.
SHA: https://github.com/cockroachdb/cockroach/commits/db9c1217a6967fcac2d135cf0f24a4265dc76d77
Parameters:
To repro, try:
Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1310296&tab=buildLog