cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30.07k stars 3.8k forks source link

roachtest: jepsen/register/strobe-skews failed #40701

Closed cockroach-teamcity closed 5 years ago

cockroach-teamcity commented 5 years ago

SHA: https://github.com/cockroachdb/cockroach/commits/991282eacbbe1315fde694be9785ad8f6fa929d3

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=jepsen/register/strobe-skews PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1481778&tab=buildLog

The test failed on branch=release-2.1, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190912-1481778/jepsen/register/strobe-skews/run_1
    jepsen.go:256,jepsen.go:322,test_runner.go:688: exit status 1
nvanbenschoten commented 5 years ago
ERROR [2019-09-12 08:27:12,625] main - jepsen.cli Oh jeez, I'm sorry, Jepsen broke. Here's why:
java.lang.InterruptedException: null
    at java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:211) ~[na:1.8.0_222]
    at java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:362) ~[na:1.8.0_222]
    at jepsen.generator.Synchronize.op(generator.clj:667) ~[jepsen-0.1.9-SNAPSHOT.jar:na]
    at jepsen.generator.Concat.op(generator.clj:609) ~[jepsen-0.1.9-SNAPSHOT.jar:na]
    at jepsen.generator$op_and_validate.invokeStatic(generator.clj:34) ~[jepsen-0.1.9-SNAPSHOT.jar:na]
    at jepsen.generator$op_and_validate.invoke(generator.clj:30) ~[jepsen-0.1.9-SNAPSHOT.jar:na]
    at jepsen.core.ClientWorker.run_worker_BANG_(core.clj:361) ~[jepsen-0.1.9-SNAPSHOT.jar:na]
    at jepsen.core$do_worker_BANG_.invokeStatic(core.clj:175) ~[jepsen-0.1.9-SNAPSHOT.jar:na]
    at jepsen.core$do_worker_BANG_.invoke(core.clj:162) ~[jepsen-0.1.9-SNAPSHOT.jar:na]
    at jepsen.core$run_workers_BANG_$fn__4599$fn__4600.invoke(core.clj:228) ~[jepsen-0.1.9-SNAPSHOT.jar:na]
    at clojure.lang.AFn.applyToHelper(AFn.java:152) ~[clojure-1.8.0.jar:na]
    at clojure.lang.AFn.applyTo(AFn.java:144) ~[clojure-1.8.0.jar:na]
    at clojure.core$apply.invokeStatic(core.clj:646) ~[clojure-1.8.0.jar:na]
    at clojure.core$with_bindings_STAR_.invokeStatic(core.clj:1881) ~[clojure-1.8.0.jar:na]
    at clojure.core$with_bindings_STAR_.doInvoke(core.clj:1881) ~[clojure-1.8.0.jar:na]
    at clojure.lang.RestFn.invoke(RestFn.java:425) ~[clojure-1.8.0.jar:na]
    at clojure.lang.AFn.applyToHelper(AFn.java:156) ~[clojure-1.8.0.jar:na]
    at clojure.lang.RestFn.applyTo(RestFn.java:132) ~[clojure-1.8.0.jar:na]
    at clojure.core$apply.invokeStatic(core.clj:650) ~[clojure-1.8.0.jar:na]
    at clojure.core$bound_fn_STAR_$fn__4671.doInvoke(core.clj:1911) ~[clojure-1.8.0.jar:na]
    at clojure.lang.RestFn.invoke(RestFn.java:397) ~[clojure-1.8.0.jar:na]
    at clojure.lang.AFn.run(AFn.java:22) ~[clojure-1.8.0.jar:na]
    at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_222]