cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30.07k stars 3.8k forks source link

roachtest: jepsen-batch3/register/majority-ring failed #31053

Closed cockroach-teamcity closed 6 years ago

cockroach-teamcity commented 6 years ago

SHA: https://github.com/cockroachdb/cockroach/commits/32346197937265bbc71dd3629b1137df9bf16fb4

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stressrace instead of stress and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
make stress TESTS=jepsen-batch3/register/majority-ring PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-stderr=false -maxtime 20m -timeout 10m'

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=948285&tab=buildLog

The test failed on master:
    test.go:500,cluster.go:897,jepsen.go:96,jepsen.go:121,jepsen.go:288: /home/agent/work/.go/bin/roachprod run teamcity-948285-jepsen-batch3:6 -- sh -c "sudo apt-get -qqy install openjdk-8-jre openjdk-8-jre-headless libjna-java gnuplot > /dev/null 2>&1" returned:
        stderr:

        stdout:
        Error:  exit status 100
        : exit status 1
    test.go:500,cluster.go:897,jepsen.go:74,asm_amd64.s:573,panic.go:377,test.go:501,cluster.go:897,jepsen.go:96,jepsen.go:121,jepsen.go:288: test already failed
tbg commented 6 years ago

apt-get failure. Note #31052 though, I bet it's the same.

bdarnell commented 6 years ago

Yeah, probably the same. But it does show that the attempt we made at capturing the logs didn't work (because it just leaves the log files on the machine but the test errors out before it captures the logs from this location). Fixing that in #31069