roachtest: hibernate failed

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/e6366f3ac39652a763f38948fccf4b2dab363034

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1347608&tab=buildLog

The test failed on branch=release-19.1, cloud=gce:
    hibernate.go:311,hibernate.go:322,test.go:1251: 
        Tests run on Cockroach v19.1.2-14-ge6366f3
        Tests run against Hibernate 5.4.3
        8596 Total Tests Run
        8270 tests passed
        326 tests failed
        0 tests passed unexpectedly
        3 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_1) is available in the artifacts' hibernate log

[tbg]

@jordanlewis not sure why this failed again, maybe just got a stale sha? Your turn

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/1bf58a7bdcaef9d2e376aee8311d46143048651a

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1347590&tab=buildLog

The test failed on branch=master, cloud=gce:
    hibernate.go:311,hibernate.go:322,test.go:1251: 
        Tests run on Cockroach v19.2.0-alpha.20190606-224-g1bf58a7
        Tests run against Hibernate 5.4.3
        8596 Total Tests Run
        8284 tests passed
        312 tests failed
        2 tests passed unexpectedly
        3 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_2) is available in the artifacts' hibernate log

[jordanlewis]

Hmm...

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/f6d75a54f2cdf21723b924e143963f75816272df

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1349751&tab=buildLog

The test failed on branch=master, cloud=gce:
    hibernate.go:311,hibernate.go:322,test.go:1251: 
        Tests run on Cockroach v19.2.0-alpha.20190606-248-gf6d75a5
        Tests run against Hibernate 5.4.3
        8596 Total Tests Run
        8282 tests passed
        314 tests failed
        2 tests passed unexpectedly
        2 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_2) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/02056538e63bd33adbc24efee05ff94385c19fc8

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1351907&tab=buildLog

The test failed on branch=master, cloud=gce:
    hibernate.go:311,hibernate.go:322,test.go:1251: 
        Tests run on Cockroach v19.2.0-alpha.20190606-268-g0205653
        Tests run against Hibernate 5.4.3
        8213 Total Tests Run
        7709 tests passed
        504 tests failed
        2 tests passed unexpectedly
        194 tests failed unexpectedly
        2 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_2) is available in the artifacts' hibernate log
    cluster.go:1035,context.go:127,cluster.go:1024,asm_amd64.s:522,panic.go:397,test.go:785,test.go:775,hibernate.go:311,hibernate.go:322,test.go:1251: dead node detection: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod monitor teamcity-1351907-hibernate --oneshot --ignore-empty-nodes: exit status 1 1: dead
        Error:  1: dead

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/5f358ed804af05f8c4b404efc4d8a282d8e0916c

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1360435&tab=buildLog

The test failed on branch=master, cloud=gce:
    hibernate.go:309,hibernate.go:320,test.go:1249: 
        Tests run on Cockroach v19.2.0-alpha.20190606-362-g5f358ed
        Tests run against Hibernate 5.4.3
        8596 Total Tests Run
        8282 tests passed
        314 tests failed
        2 tests passed unexpectedly
        2 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_2) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/5f358ed804af05f8c4b404efc4d8a282d8e0916c

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1361643&tab=buildLog

The test failed on branch=master, cloud=gce:
    hibernate.go:309,hibernate.go:320,test.go:1249: 
        Tests run on Cockroach v19.2.0-alpha.20190606-362-g5f358ed
        Tests run against Hibernate 5.4.3
        8596 Total Tests Run
        8282 tests passed
        314 tests failed
        2 tests passed unexpectedly
        2 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_2) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/90841a6559df9d9a4724e1d30490951bbdb811b4

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1364443&tab=buildLog

The test failed on branch=provisional_201906271846_v19.2.0-alpha.20190701, cloud=gce:
    hibernate.go:309,hibernate.go:320,test.go:1249: 
        Tests run on Cockroach v19.2.0-alpha.20190606-423-g90841a6
        Tests run against Hibernate 5.4.3
        8596 Total Tests Run
        8282 tests passed
        314 tests failed
        2 tests passed unexpectedly
        2 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_2) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/537767ac9daa52b0026bb957d7010e3b88b61071

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1364821&tab=buildLog

The test failed on branch=master, cloud=gce:
    hibernate.go:309,hibernate.go:320,test.go:1249: 
        Tests run on Cockroach v19.2.0-alpha.20190606-453-g537767a
        Tests run against Hibernate 5.4.3
        8596 Total Tests Run
        8282 tests passed
        314 tests failed
        2 tests passed unexpectedly
        2 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_2) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/5968fa55d48966866eac1cb744c61d0aa6284ae7

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1367354&tab=buildLog

The test failed on branch=provisional_201906292257_v19.2.0-alpha.20190701, cloud=gce:
    hibernate.go:309,hibernate.go:320,test.go:1249: 
        Tests run on Cockroach v19.2.0-alpha.20190606-319-g5968fa5
        Tests run against Hibernate 5.4.3
        8596 Total Tests Run
        8282 tests passed
        314 tests failed
        2 tests passed unexpectedly
        2 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_2) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/86154ae6ae36e286883d8a6c9a4111966198201d

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1367379&tab=buildLog

The test failed on branch=master, cloud=gce:
    hibernate.go:309,hibernate.go:320,test.go:1249: 
        Tests run on Cockroach v19.2.0-alpha.20190606-472-g86154ae
        Tests run against Hibernate 5.4.3
        8596 Total Tests Run
        8282 tests passed
        314 tests failed
        2 tests passed unexpectedly
        2 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_2) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/ca1ef4d4f8296b213c0b2b140f16e4a97931e6e7

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1368144&tab=buildLog

The test failed on branch=master, cloud=gce:
    hibernate.go:309,hibernate.go:320,test.go:1249: 
        Tests run on Cockroach v19.2.0-alpha.20190606-474-gca1ef4d
        Tests run against Hibernate 5.4.3
        8049 Total Tests Run
        7425 tests passed
        624 tests failed
        2 tests passed unexpectedly
        318 tests failed unexpectedly
        6 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_2) is available in the artifacts' hibernate log
    cluster.go:1033,context.go:122,cluster.go:1022,panic.go:406,test.go:783,test.go:773,hibernate.go:309,hibernate.go:320,test.go:1249: dead node detection: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod monitor teamcity-1368144-hibernate --oneshot --ignore-empty-nodes: exit status 1 1: dead
        Error:  1: dead

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/2c865eeb3e3b244468ffc509a62778bd1f46740f

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1370685&tab=buildLog

The test failed on branch=provisional_201907021644_v19.2.0-alpha.20190701, cloud=gce:
    hibernate.go:309,hibernate.go:320,test.go:1249: 
        Tests run on Cockroach v19.2.0-alpha.20190606-321-g2c865ee
        Tests run against Hibernate 5.4.3
        8596 Total Tests Run
        8282 tests passed
        314 tests failed
        2 tests passed unexpectedly
        2 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_2) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/4878cb3e960dc26f0946e527e1836f27b3304c97

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1373334&tab=buildLog

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190704-1373334/hibernate/run_1
    hibernate.go:310,hibernate.go:321,test_runner.go:681: 
        Tests run on Cockroach v19.2.0-alpha.20190606-556-g4878cb3
        Tests run against Hibernate 5.4.3
        8596 Total Tests Run
        8282 tests passed
        314 tests failed
        2 tests passed unexpectedly
        2 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_2) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/a2f104c467ee1553a875b45f1e16f8cfb3e3195d

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1374703&tab=buildLog

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190705-1374703/hibernate/run_1
    hibernate.go:310,hibernate.go:321,test_runner.go:681: 
        Tests run on Cockroach v19.2.0-alpha.20190606-559-ga2f104c
        Tests run against Hibernate 5.4.3
        8596 Total Tests Run
        8282 tests passed
        314 tests failed
        2 tests passed unexpectedly
        2 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_2) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/9322e07476de447799c5d3011eb2874930ee2993

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1375546&tab=buildLog

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190706-1375546/hibernate/run_1
    hibernate.go:310,hibernate.go:321,test_runner.go:681: 
        Tests run on Cockroach v19.2.0-alpha.20190606-563-g9322e07
        Tests run against Hibernate 5.4.3
        8596 Total Tests Run
        8282 tests passed
        314 tests failed
        2 tests passed unexpectedly
        2 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_2) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/8c6fdc64908a13291e4ddc5d233bbbaa379e71a2

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1378458&tab=buildLog

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190709-1378458/hibernate/run_1
    hibernate.go:310,hibernate.go:321,test_runner.go:670: 
        Tests run on Cockroach v19.2.0-alpha.20190606-625-g8c6fdc6
        Tests run against Hibernate 5.4.3
        8596 Total Tests Run
        8282 tests passed
        314 tests failed
        2 tests passed unexpectedly
        2 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_2) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/813b0146e763e8f44c4cb45e498e68e6ace9bc2d

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1380768&tab=buildLog

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190710-1380768/hibernate/run_1
    hibernate.go:310,hibernate.go:321,test_runner.go:678: 
        Tests run on Cockroach v19.2.0-alpha.20190606-664-g813b014
        Tests run against Hibernate 5.4.3
        8596 Total Tests Run
        8285 tests passed
        311 tests failed
        5 tests passed unexpectedly
        2 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_2) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/1ca35fc4a0e2665e7f6efd945e65a0db97984fa7

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1396096&tab=buildLog

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190719-1396096/hibernate/run_1
    hibernate.go:310,hibernate.go:321,test_runner.go:691: 
        Tests run on Cockroach v19.2.0-alpha.20190606-877-g1ca35fc
        Tests run against Hibernate 5.4.3
        8596 Total Tests Run
        8281 tests passed
        315 tests failed
        5 tests passed unexpectedly
        6 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_2) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/22d48caaa7d39efdcef7b3c87a99fc421e1473af

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1397412&tab=buildLog

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190720-1397412/hibernate/run_1
    hibernate.go:310,hibernate.go:321,test_runner.go:691: 
        Tests run on Cockroach v19.2.0-alpha.20190606-918-g22d48ca
        Tests run against Hibernate 5.4.3
        8596 Total Tests Run
        8281 tests passed
        315 tests failed
        5 tests passed unexpectedly
        6 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_2) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/7dab0dcfd37c389af357c302c073b9611b5ada25

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1398203&tab=buildLog

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190721-1398203/hibernate/run_1
    hibernate.go:310,hibernate.go:321,test_runner.go:691: 
        Tests run on Cockroach v19.2.0-alpha.20190606-923-g7dab0dc
        Tests run against Hibernate 5.4.3
        8596 Total Tests Run
        8281 tests passed
        315 tests failed
        5 tests passed unexpectedly
        6 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_2) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/1ad0ecc8cbddf82c9fedb5a5c5e533e72a657ff7

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1399000&tab=buildLog

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190722-1399000/hibernate/run_1
    hibernate.go:310,hibernate.go:321,test_runner.go:691: 
        Tests run on Cockroach v19.2.0-alpha.20190606-925-g1ad0ecc
        Tests run against Hibernate 5.4.3
        8596 Total Tests Run
        8281 tests passed
        315 tests failed
        5 tests passed unexpectedly
        6 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_2) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/7111a67b2ea3a19c2f312f8d214b8823f431cac0

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1400942&tab=buildLog

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190723-1400942/hibernate/run_1
    hibernate.go:310,hibernate.go:321,test_runner.go:691: 
        Tests run on Cockroach v19.2.0-alpha.20190606-963-g7111a67
        Tests run against Hibernate 5.4.3
        8596 Total Tests Run
        8279 tests passed
        317 tests failed
        5 tests passed unexpectedly
        8 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_2) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/86eab2ff0a1a4c2d9b5f7e7a45deda74c98c6c37

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1402541&tab=buildLog

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190724-1402541/hibernate/run_1
    hibernate.go:310,hibernate.go:321,test_runner.go:691: 
        Tests run on Cockroach v19.2.0-alpha.20190606-983-g86eab2f
        Tests run against Hibernate 5.4.3
        8596 Total Tests Run
        8281 tests passed
        315 tests failed
        5 tests passed unexpectedly
        6 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_2) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/26edea51118a0e16b61748c08068bfa6f76543ca

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1404886&tab=buildLog

The test failed on branch=provisional_201907241708_v19.2.0-alpha.20190729, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190725-1404886/hibernate/run_1
    hibernate.go:310,hibernate.go:321,test_runner.go:691: 
        Tests run on Cockroach v19.2.0-alpha.20190606-997-g26edea5
        Tests run against Hibernate 5.4.3
        8596 Total Tests Run
        8281 tests passed
        315 tests failed
        5 tests passed unexpectedly
        6 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_2) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/ff04012ed2d2c0c8e30e4de106ca0a350bca8c3e

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1404856&tab=buildLog

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190725-1404856/hibernate/run_1
    hibernate.go:310,hibernate.go:321,test_runner.go:691: 
        Tests run on Cockroach v19.2.0-alpha.20190606-1007-gff04012
        Tests run against Hibernate 5.4.3
        8596 Total Tests Run
        8281 tests passed
        315 tests failed
        5 tests passed unexpectedly
        6 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_2) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/9078c4e63c1bff1c3d220ee216000b0903dd4d65

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1406479&tab=buildLog

The test failed on branch=provisional_201907252112_v19.2.0-alpha.20190729, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190726-1406479/hibernate/run_1
    hibernate.go:310,hibernate.go:321,test_runner.go:691: 
        Tests run on Cockroach v19.2.0-alpha.20190606-998-g9078c4e
        Tests run against Hibernate 5.4.3
        8596 Total Tests Run
        8281 tests passed
        315 tests failed
        5 tests passed unexpectedly
        6 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_2) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/cfdaadc3514e7e8660f6c009ba159fdfd604f0a8

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1409070&tab=buildLog

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190727-1409070/hibernate/run_1
    hibernate.go:310,hibernate.go:321,test_runner.go:691: 
        Tests run on Cockroach v19.2.0-alpha.20190606-1046-gcfdaadc
        Tests run against Hibernate 5.4.3
        8596 Total Tests Run
        8281 tests passed
        315 tests failed
        5 tests passed unexpectedly
        6 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_2) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/008680701c3aaf3f5f621dca781819096178cff4

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1411175&tab=buildLog

The test failed on branch=release-19.1, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190730-1411175/hibernate/run_1
    hibernate.go:310,hibernate.go:321,test_runner.go:691: 
        Tests run on Cockroach v19.1.3-22-g0086807
        Tests run against Hibernate 5.4.4
        9527 Total Tests Run
        9175 tests passed
        352 tests failed
        3 tests passed unexpectedly
        29 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_1) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/1b682fa1ed7fa7f24c6518efa9bfc10ede816749

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1413406&tab=buildLog

The test failed on branch=release-19.1, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190731-1413406/hibernate/run_1
    hibernate.go:310,hibernate.go:321,test_runner.go:691: 
        Tests run on Cockroach v19.1.3-24-g1b682fa
        Tests run against Hibernate 5.4.4
        9527 Total Tests Run
        9175 tests passed
        352 tests failed
        3 tests passed unexpectedly
        29 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_1) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/92fef12128c997233d985d1c19e11faac005073f

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1413388&tab=buildLog

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190731-1413388/hibernate/run_1
    hibernate.go:310,hibernate.go:321,test_runner.go:691: 
        Tests run on Cockroach v19.2.0-alpha.20190606-1119-g92fef12
        Tests run against Hibernate 5.4.4
        9467 Total Tests Run
        9061 tests passed
        406 tests failed
        5 tests passed unexpectedly
        97 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_2) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/da56c792e968574b8f1d9ef3fdb45d56a530221a

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1415578&tab=buildLog

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190801-1415578/hibernate/run_1
    hibernate.go:310,hibernate.go:321,test_runner.go:691: 
        Tests run on Cockroach v19.2.0-alpha.20190606-1131-gda56c79
        Tests run against Hibernate 5.4.4
        9468 Total Tests Run
        9052 tests passed
        416 tests failed
        5 tests passed unexpectedly
        107 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_2) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/51a6fdedf0ce1d1329d40d801a7deaf8206b6b07

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1420118&tab=buildLog

The test failed on branch=release-19.1, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190803-1420118/hibernate/run_1
    hibernate.go:310,hibernate.go:321,test_runner.go:691: 
        Tests run on Cockroach v19.1.3-31-g51a6fde
        Tests run against Hibernate 5.4.4
        9527 Total Tests Run
        9175 tests passed
        352 tests failed
        3 tests passed unexpectedly
        29 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_1) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/3b9a95bd7eb2cfa6d544fe7217852a85ec3b76f4

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1422703&tab=buildLog

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190805-1422703/hibernate/run_1
    hibernate.go:310,hibernate.go:321,test_runner.go:691: 
        Tests run on Cockroach v19.2.0-alpha.20190606-1213-g3b9a95b
        Tests run against Hibernate 5.4.4
        6318 Total Tests Run
        3857 tests passed
        2461 tests failed
        1 test passed unexpectedly
        2259 tests failed unexpectedly
        111 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_2) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/3db89b230b0c41e399354cbeb78c1e82c8e30004

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1424320&tab=buildLog

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190806-1424320/hibernate/run_1
    hibernate.go:310,hibernate.go:321,test_runner.go:691: 
        Tests run on Cockroach v19.2.0-alpha.20190606-1259-g3db89b2
        Tests run against Hibernate 5.4.4
        5394 Total Tests Run
        3419 tests passed
        1975 tests failed
        1 test passed unexpectedly
        1872 tests failed unexpectedly
        210 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_2) is available in the artifacts' hibernate log

[jordanlewis]

Hmm... this doesn't look good. @cockroachdb/sql-execution we need to investigate this - I think probably there's a crash happening since we merged the auto setting, but the artifacts aren't getting uploaded because a core dump was made that exceeds the maximum artifact size?

We need to run this one by hand and do some forensics.

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/a850466dfc4f8eed9e7f758a61f60b120798410f

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1426564&tab=buildLog

The test failed on branch=release-19.1, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190807-1426564/hibernate/run_1
    hibernate.go:309,hibernate.go:320,test_runner.go:691: 
        Tests run on Cockroach v19.1.3-33-ga850466
        Tests run against Hibernate 5.4.4
        9527 Total Tests Run
        9175 tests passed
        352 tests failed
        3 tests passed unexpectedly
        29 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_1) is available in the artifacts' hibernate log

[jordanlewis]

Found the panic:

* ERROR: [n1,client=127.0.0.1:46738,user=root] a SQL panic has occurred while executing "SELECT mv.enumimplicitoverridestring FROM map_value AS mv WHERE mv.id = $1": runtime error: slice bounds out of range
*
*
* ERROR: [n1,client=127.0.0.1:46738,user=root] a panic has occurred!
*
panic while executing 1 statements: SELECT _._ FROM _ AS _ WHERE _._ = $1

goroutine 51691834 [running]:
runtime/debug.Stack(0x43fa240, 0xc033569480, 0x3)
    /usr/local/go/src/runtime/debug/stack.go:24 +0x9d
github.com/cockroachdb/cockroach/pkg/util/log.ReportPanic(0x43fa240, 0xc033569480, 0xc000656000, 0x39ac220, 0xc0298c5ef0, 0x1)
    /go/src/github.com/cockroachdb/cockroach/pkg/util/log/crash_reporting.go:217 +0xa3
github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).closeWrapper(0xc01062fb00, 0x43fa240, 0xc033569480, 0x3718820, 0x65e0f00)
    /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:720 +0x2a0
github.com/cockroachdb/cockroach/pkg/sql.(*Server).ServeConn.func1(0xc01062fb00, 0x43fa240, 0xc033569480)
    /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:440 +0x61
panic(0x3718820, 0x65e0f00)
    /usr/local/go/src/runtime/panic.go:522 +0x1b5
github.com/cockroachdb/cockroach/pkg/sql/exec.CatchVectorizedRuntimeError.func1(0xc03d11e1d0)
    /go/src/github.com/cockroachdb/cockroach/pkg/sql/exec/error.go:71 +0x6d7
panic(0x3718820, 0x65e0f00)
    /usr/local/go/src/runtime/panic.go:522 +0x1b5
github.com/cockroachdb/cockroach/pkg/util/encoding.DecodeUntaggedBytesValue(0xc0ab1ec0b0, 0x9, 0x9, 0x0, 0x1, 0x0, 0x1, 0xc03d11d380, 0xf27a5d, 0x43fa240, ...)
    /go/src/github.com/cockroachdb/cockroach/pkg/util/encoding/encoding.go:2034 +0x11e
github.com/cockroachdb/cockroach/pkg/sql/colencoding.decodeUntaggedDatumToCol(0x445e320, 0xc017e27480, 0x0, 0xc046754e48, 0xc0ab1ec0b0, 0x9, 0x9, 0x0, 0x43fa240, 0xc017e27240, ...)
    /go/src/github.com/cockroachdb/cockroach/pkg/sql/colencoding/value_encoding.go:60 +0x719
github.com/cockroachdb/cockroach/pkg/sql/colencoding.DecodeTableValueToCol(0x445e320, 0xc017e27480, 0x0, 0x6, 0x2, 0xc046754e48, 0xc0ab1ec0ae, 0xb, 0xb, 0x0, ...)
    /go/src/github.com/cockroachdb/cockroach/pkg/sql/colencoding/value_encoding.go:36 +0xbd
github.com/cockroachdb/cockroach/pkg/sql/row.(*CFetcher).processValueBytes(0xc037dce500, 0x43fa240, 0xc017e275c0, 0xc037dce500, 0xc0ab1ec0ad, 0xc, 0xc, 0x0, 0x0, 0x0, ...)
    /go/src/github.com/cockroachdb/cockroach/pkg/sql/row/cfetcher.go:1059 +0x40f
github.com/cockroachdb/cockroach/pkg/sql/row.(*CFetcher).processValueTuple(...)
    /go/src/github.com/cockroachdb/cockroach/pkg/sql/row/cfetcher.go:1083
github.com/cockroachdb/cockroach/pkg/sql/row.(*CFetcher).processValue(0xc037dce500, 0x43fa240, 0xc017e275c0, 0x0, 0x0, 0x0, 0x21, 0xc0ab1ec0a8, 0x11, 0x11)
    /go/src/github.com/cockroachdb/cockroach/pkg/sql/row/cfetcher.go:845 +0x826
github.com/cockroachdb/cockroach/pkg/sql/row.(*CFetcher).NextBatch(0xc037dce500, 0x43fa240, 0xc017e275c0, 0xc017e20000, 0xc03d11dbe8, 0x1ec975e, 0x0)
    /go/src/github.com/cockroachdb/cockroach/pkg/sql/row/cfetcher.go:641 +0x6c1
github.com/cockroachdb/cockroach/pkg/sql/distsqlrun.(*colBatchScan).Next(0xc049aeb6d0, 0x43fa240, 0xc017e275c0, 0xc071519350, 0xc0298c5bc0)
    /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsqlrun/colbatch_scan.go:58 +0x47
github.com/cockroachdb/cockroach/pkg/sql/exec.(*CancelChecker).Next(0xc07e59c780, 0x43fa240, 0xc017e275c0, 0x0, 0x0)
    /go/src/github.com/cockroachdb/cockroach/pkg/sql/exec/cancel_checker.go:40 +0x69
github.com/cockroachdb/cockroach/pkg/sql/exec.(*simpleProjectOp).Next(0xc07e59c7a0, 0x43fa240, 0xc017e275c0, 0x0, 0x0)
    /go/src/github.com/cockroachdb/cockroach/pkg/sql/exec/simple_project.go:90 +0x4b
github.com/cockroachdb/cockroach/pkg/sql/distsqlrun.(*materializer).next(0xc02cfa1800, 0x73db6f, 0xc000000008, 0xc04cf43280, 0xc03d11e1b0)
    /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsqlrun/materializer.go:150 +0x451
github.com/cockroachdb/cockroach/pkg/sql/distsqlrun.(*materializer).nextAdapter(...)
    /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsqlrun/materializer.go:141
github.com/cockroachdb/cockroach/pkg/sql/exec.CatchVectorizedRuntimeError(0xc03d11e1e0, 0x0, 0x0)
    /go/src/github.com/cockroachdb/cockroach/pkg/sql/exec/error.go:80 +0x55
github.com/cockroachdb/cockroach/pkg/sql/distsqlrun.(*materializer).Next(0xc02cfa1800, 0x43fa240, 0xc017e275c0, 0xc071519360, 0x43fa300)
    /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsqlrun/materializer.go:187 +0x4d
github.com/cockroachdb/cockroach/pkg/sql/distsqlrun.Run(0x43fa240, 0xc017e275c0, 0x4406000, 0xc02cfa1800, 0x43d5c40, 0xc0170a0000)
    /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsqlrun/base.go:171 +0x35
github.com/cockroachdb/cockroach/pkg/sql/distsqlrun.(*ProcessorBase).Run(0xc02cfa1800, 0x43fa300, 0xc0298c5a40)
    /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsqlrun/processors.go:793 +0x92
github.com/cockroachdb/cockroach/pkg/sql/distsqlrun.(*Flow).Run(0xc01f5aeea0, 0x43fa300, 0xc0298c5a40, 0x3cc8898, 0x0, 0x0)
    /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsqlrun/flow.go:578 +0x1e9
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).Run(0xc0001bc000, 0xc00038b2c0, 0xc0062e3950, 0xc03d11e878, 0xc0170a0000, 0xc01062ff28, 0x0, 0x0)
    /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:328 +0x368
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).PlanAndRun(0xc0001bc000, 0x43fa300, 0xc0298c5530, 0xc01062ff28, 0xc00038b2c0, 0xc0062e3950, 0x43fc040, 0xc01f5aed80, 0xc0170a0000, 0xc01062fcd8)
    /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:937 +0x217
github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execWithDistSQLEngine(0xc01062fb00, 0x43fa300, 0xc0298c5530, 0xc01062fe50, 0x3, 0x7fea30f3cda8, 0xc02a7ca168, 0xc03d11ea00, 0x0, 0x0, ...)
    /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:902 +0x370
github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).dispatchToExecutionEngine(0xc01062fb00, 0x43fa300, 0xc0298c5530, 0xc01062fe50, 0x7fea30f3cda8, 0xc02a7ca168, 0x0, 0x0)
    /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:734 +0x6e5
github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmtInOpenState(0xc01062fb00, 0x43fa300, 0xc0298c5530, 0x43ffa00, 0xc017e26a80, 0xc01ebe8e5f, 0x45, 0x1, 0x1, 0xc07e59c1e0, ...)
    /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:417 +0xb64
github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmt(0xc01062fb00, 0x43fa300, 0xc0298c5530, 0x43ffa00, 0xc017e26a80, 0xc01ebe8e5f, 0x45, 0x1, 0x1, 0xc07e59c1e0, ...)
    /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:99 +0x4ec
github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execCmd(0xc01062fb00, 0x43fa240, 0xc00cb876c0, 0x0, 0x0)
    /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:1273 +0xee8
github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).run(0xc01062fb00, 0x43fa240, 0xc033569480, 0xc0000330c0, 0x5400, 0x15000, 0xc000033158, 0xc015641010, 0x0, 0x0)
    /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:1140 +0x1a3
github.com/cockroachdb/cockroach/pkg/sql.(*Server).ServeConn(0xc0001b6480, 0x43fa240, 0xc033569480, 0xc01062fb00, 0x5400, 0x15000, 0xc000033158, 0xc015641010, 0x0, 0x0)
    /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:442 +0xce
github.com/cockroachdb/cockroach/pkg/sql/pgwire.(*conn).processCommandsAsync.func1(0xc03d76b619, 0xc0377a0160, 0x43fa240, 0xc033569480, 0xc015641010, 0xc0001b6480, 0xc02a7c9b00, 0x43fdf80, 0xc0377a0140, 0xc01a310f60, ...)
    /go/src/github.com/cockroachdb/cockroach/pkg/sql/pgwire/conn.go:584 +0x21c
created by github.com/cockroachdb/cockroach/pkg/sql/pgwire.(*conn).processCommandsAsync
    /go/src/github.com/cockroachdb/cockroach/pkg/sql/pgwire/conn.go:513 +0x17b

[BramGruneir]

Nice. Even while failing these tests provide insight.

On Wed, Aug 7, 2019 at 12:53 PM Jordan Lewis notifications@github.com wrote:

Found the panic:

• ERROR: [n1,client=127.0.0.1:46738,user=root] a SQL panic has occurred while executing "SELECT mv.enumimplicitoverridestring FROM map_value AS mv WHERE mv.id = $1": runtime error: slice bounds out of range
• ERROR: [n1,client=127.0.0.1:46738,user=root] a panic has occurred!

• panic while executing 1 statements: SELECT . FROM AS WHERE . = $1

goroutine 51691834 [running]: runtime/debug.Stack(0x43fa240, 0xc033569480, 0x3) /usr/local/go/src/runtime/debug/stack.go:24 +0x9dgithub.com/cockroachdb/cockroach/pkg/util/log.ReportPanic(0x43fa240, 0xc033569480, 0xc000656000, 0x39ac220, 0xc0298c5ef0, 0x1) /go/src/github.com/cockroachdb/cockroach/pkg/util/log/crash_reporting.go:217 +0xa3github.com/cockroachdb/cockroach/pkg/sql.(connExecutor).closeWrapper(0xc01062fb00, 0x43fa240, 0xc033569480, 0x3718820, 0x65e0f00) /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:720 +0x2a0github.com/cockroachdb/cockroach/pkg/sql.(Server).ServeConn.func1(0xc01062fb00, 0x43fa240, 0xc033569480) /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:440 +0x61 panic(0x3718820, 0x65e0f00) /usr/local/go/src/runtime/panic.go:522 +0x1b5github.com/cockroachdb/cockroach/pkg/sql/exec.CatchVectorizedRuntimeError.func1(0xc03d11e1d0) /go/src/github.com/cockroachdb/cockroach/pkg/sql/exec/error.go:71 +0x6d7 panic(0x3718820, 0x65e0f00) /usr/local/go/src/runtime/panic.go:522 +0x1b5github.com/cockroachdb/cockroach/pkg/util/encoding.DecodeUntaggedBytesValue(0xc0ab1ec0b0, 0x9, 0x9, 0x0, 0x1, 0x0, 0x1, 0xc03d11d380, 0xf27a5d, 0x43fa240, ...) /go/src/github.com/cockroachdb/cockroach/pkg/util/encoding/encoding.go:2034 +0x11egithub.com/cockroachdb/cockroach/pkg/sql/colencoding.decodeUntaggedDatumToCol(0x445e320, 0xc017e27480, 0x0, 0xc046754e48, 0xc0ab1ec0b0, 0x9, 0x9, 0x0, 0x43fa240, 0xc017e27240, ...) /go/src/github.com/cockroachdb/cockroach/pkg/sql/colencoding/value_encoding.go:60 +0x719github.com/cockroachdb/cockroach/pkg/sql/colencoding.DecodeTableValueToCol(0x445e320, 0xc017e27480, 0x0, 0x6, 0x2, 0xc046754e48, 0xc0ab1ec0ae, 0xb, 0xb, 0x0, ...) /go/src/github.com/cockroachdb/cockroach/pkg/sql/colencoding/value_encoding.go:36 +0xbdgithub.com/cockroachdb/cockroach/pkg/sql/row.(CFetcher).processValueBytes(0xc037dce500, 0x43fa240, 0xc017e275c0, 0xc037dce500, 0xc0ab1ec0ad, 0xc, 0xc, 0x0, 0x0, 0x0, ...) /go/src/github.com/cockroachdb/cockroach/pkg/sql/row/cfetcher.go:1059 +0x40fgithub.com/cockroachdb/cockroach/pkg/sql/row.(CFetcher).processValueTuple(...) /go/src/github.com/cockroachdb/cockroach/pkg/sql/row/cfetcher.go:1083github.com/cockroachdb/cockroach/pkg/sql/row.(CFetcher).processValue(0xc037dce500, 0x43fa240, 0xc017e275c0, 0x0, 0x0, 0x0, 0x21, 0xc0ab1ec0a8, 0x11, 0x11) /go/src/github.com/cockroachdb/cockroach/pkg/sql/row/cfetcher.go:845 +0x826github.com/cockroachdb/cockroach/pkg/sql/row.(CFetcher).NextBatch(0xc037dce500, 0x43fa240, 0xc017e275c0, 0xc017e20000, 0xc03d11dbe8, 0x1ec975e, 0x0) /go/src/github.com/cockroachdb/cockroach/pkg/sql/row/cfetcher.go:641 +0x6c1github.com/cockroachdb/cockroach/pkg/sql/distsqlrun.(colBatchScan).Next(0xc049aeb6d0, 0x43fa240, 0xc017e275c0, 0xc071519350, 0xc0298c5bc0) /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsqlrun/colbatch_scan.go:58 +0x47github.com/cockroachdb/cockroach/pkg/sql/exec.(CancelChecker).Next(0xc07e59c780, 0x43fa240, 0xc017e275c0, 0x0, 0x0) /go/src/github.com/cockroachdb/cockroach/pkg/sql/exec/cancel_checker.go:40 +0x69github.com/cockroachdb/cockroach/pkg/sql/exec.(simpleProjectOp).Next(0xc07e59c7a0, 0x43fa240, 0xc017e275c0, 0x0, 0x0) /go/src/github.com/cockroachdb/cockroach/pkg/sql/exec/simple_project.go:90 +0x4bgithub.com/cockroachdb/cockroach/pkg/sql/distsqlrun.(materializer).next(0xc02cfa1800, 0x73db6f, 0xc000000008, 0xc04cf43280, 0xc03d11e1b0) /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsqlrun/materializer.go:150 +0x451github.com/cockroachdb/cockroach/pkg/sql/distsqlrun.(materializer).nextAdapter(...) /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsqlrun/materializer.go:141github.com/cockroachdb/cockroach/pkg/sql/exec.CatchVectorizedRuntimeError(0xc03d11e1e0, 0x0, 0x0) /go/src/github.com/cockroachdb/cockroach/pkg/sql/exec/error.go:80 +0x55github.com/cockroachdb/cockroach/pkg/sql/distsqlrun.(materializer).Next(0xc02cfa1800, 0x43fa240, 0xc017e275c0, 0xc071519360, 0x43fa300) /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsqlrun/materializer.go:187 +0x4dgithub.com/cockroachdb/cockroach/pkg/sql/distsqlrun.Run(0x43fa240, 0xc017e275c0, 0x4406000, 0xc02cfa1800, 0x43d5c40, 0xc0170a0000) /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsqlrun/base.go:171 +0x35github.com/cockroachdb/cockroach/pkg/sql/distsqlrun.(ProcessorBase).Run(0xc02cfa1800, 0x43fa300, 0xc0298c5a40) /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsqlrun/processors.go:793 +0x92github.com/cockroachdb/cockroach/pkg/sql/distsqlrun.(Flow).Run(0xc01f5aeea0, 0x43fa300, 0xc0298c5a40, 0x3cc8898, 0x0, 0x0) /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsqlrun/flow.go:578 +0x1e9github.com/cockroachdb/cockroach/pkg/sql.(DistSQLPlanner).Run(0xc0001bc000, 0xc00038b2c0, 0xc0062e3950, 0xc03d11e878, 0xc0170a0000, 0xc01062ff28, 0x0, 0x0) /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:328 +0x368github.com/cockroachdb/cockroach/pkg/sql.(DistSQLPlanner).PlanAndRun(0xc0001bc000, 0x43fa300, 0xc0298c5530, 0xc01062ff28, 0xc00038b2c0, 0xc0062e3950, 0x43fc040, 0xc01f5aed80, 0xc0170a0000, 0xc01062fcd8) /go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:937 +0x217github.com/cockroachdb/cockroach/pkg/sql.(connExecutor).execWithDistSQLEngine(0xc01062fb00, 0x43fa300, 0xc0298c5530, 0xc01062fe50, 0x3, 0x7fea30f3cda8, 0xc02a7ca168, 0xc03d11ea00, 0x0, 0x0, ...) /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:902 +0x370github.com/cockroachdb/cockroach/pkg/sql.(connExecutor).dispatchToExecutionEngine(0xc01062fb00, 0x43fa300, 0xc0298c5530, 0xc01062fe50, 0x7fea30f3cda8, 0xc02a7ca168, 0x0, 0x0) /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:734 +0x6e5github.com/cockroachdb/cockroach/pkg/sql.(connExecutor).execStmtInOpenState(0xc01062fb00, 0x43fa300, 0xc0298c5530, 0x43ffa00, 0xc017e26a80, 0xc01ebe8e5f, 0x45, 0x1, 0x1, 0xc07e59c1e0, ...) /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:417 +0xb64github.com/cockroachdb/cockroach/pkg/sql.(connExecutor).execStmt(0xc01062fb00, 0x43fa300, 0xc0298c5530, 0x43ffa00, 0xc017e26a80, 0xc01ebe8e5f, 0x45, 0x1, 0x1, 0xc07e59c1e0, ...) /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:99 +0x4ecgithub.com/cockroachdb/cockroach/pkg/sql.(connExecutor).execCmd(0xc01062fb00, 0x43fa240, 0xc00cb876c0, 0x0, 0x0) /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:1273 +0xee8github.com/cockroachdb/cockroach/pkg/sql.(connExecutor).run(0xc01062fb00, 0x43fa240, 0xc033569480, 0xc0000330c0, 0x5400, 0x15000, 0xc000033158, 0xc015641010, 0x0, 0x0) /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:1140 +0x1a3github.com/cockroachdb/cockroach/pkg/sql.(Server).ServeConn(0xc0001b6480, 0x43fa240, 0xc033569480, 0xc01062fb00, 0x5400, 0x15000, 0xc000033158, 0xc015641010, 0x0, 0x0) /go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:442 +0xcegithub.com/cockroachdb/cockroach/pkg/sql/pgwire.(conn).processCommandsAsync.func1(0xc03d76b619, 0xc0377a0160, 0x43fa240, 0xc033569480, 0xc015641010, 0xc0001b6480, 0xc02a7c9b00, 0x43fdf80, 0xc0377a0140, 0xc01a310f60, ...) /go/src/github.com/cockroachdb/cockroach/pkg/sql/pgwire/conn.go:584 +0x21c created by github.com/cockroachdb/cockroach/pkg/sql/pgwire.(*conn).processCommandsAsync /go/src/github.com/cockroachdb/cockroach/pkg/sql/pgwire/conn.go:513 +0x17b

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/cockroachdb/cockroach/issues/38288?email_source=notifications&email_token=AAMKDONSFJQASZCN57TE4ILQDL4ZTA5CNFSM4HZINZ3KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3ZBGFY#issuecomment-519181079, or mute the thread https://github.com/notifications/unsubscribe-auth/AAMKDOMQWITDDDPTHNNJNX3QDL4ZTANCNFSM4HZINZ3A .

[jordanlewis]

Repro:

root@127.0.0.1:51637/movr> CREATE TABLE map_value3 (id INT4 NOT NULL, enumdefault INT4, enumexplicit VARCHAR(255), enumexplicitordinal INT4, enumexplicitstring VARCHAR(255), enumimplicit VARCHAR(255), enumimplicitoverrideordinal INT4, enumimplicitoverridestring VARCHAR(255), enumimplicitoverrided VARCHAR(255), explicittype VARCHAR(255), implicittype VARCHAR(255), map_entity_id INT4, PRIMARY KEY (id));
CREATE TABLE

Time: 2.056ms

root@127.0.0.1:51637/movr> insert into map_value3 values(2,null,null,null,null,null,null,'VALUE_2',null,null,null,1);
INSERT 1

Time: 2.422ms

root@127.0.0.1:51637/movr> select * from map_value3;

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/51a6fdedf0ce1d1329d40d801a7deaf8206b6b07

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1428934&tab=buildLog

The test failed on branch=provisional_201908060405_v19.1.4, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190807-1428934/hibernate/run_1
    hibernate.go:309,hibernate.go:320,test_runner.go:691: 
        Tests run on Cockroach v19.1.3-31-g51a6fde
        Tests run against Hibernate 5.4.4
        9527 Total Tests Run
        9175 tests passed
        352 tests failed
        3 tests passed unexpectedly
        29 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_1) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/1051f376924f18c5782887ed824ab5ede5027493

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1429376&tab=buildLog

The test failed on branch=release-2.1, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190808-1429376/hibernate/run_1
    hibernate.go:309,hibernate.go:320,test_runner.go:691: 
        Tests run on Cockroach v2.1.8-6-g1051f37
        Tests run against Hibernate 5.4.4
        9527 Total Tests Run
        9164 tests passed
        363 tests failed
        0 tests passed unexpectedly
        29 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList2_1) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/51a6fdedf0ce1d1329d40d801a7deaf8206b6b07

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1436116&tab=buildLog

The test failed on branch=provisional_201908060405_v19.1.4, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190812-1436116/hibernate/run_1
    hibernate.go:321,hibernate.go:332,test_runner.go:691: 
        Tests run on Cockroach v19.1.3-31-g51a6fde
        Tests run against Hibernate 5.4.4
        9527 Total Tests Run
        9175 tests passed
        352 tests failed
        3 tests passed unexpectedly
        29 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_1) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/01ee0704865391599abef3bbc89f462117f8007a

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1445527&tab=buildLog

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190820-1445527/hibernate/run_1
    hibernate.go:321,hibernate.go:332,test_runner.go:673: 
        Tests run on Cockroach v19.2.0-alpha.20190606-1629-g01ee070
        Tests run against Hibernate 5.4.4
        9519 Total Tests Run
        9185 tests passed
        334 tests failed
        5 tests passed unexpectedly
        25 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_2) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/9a982e902638e116ed6a76f4fa635a0a1445d88a

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1447054&tab=buildLog

The test failed on branch=release-19.1, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190821-1447054/hibernate/run_1
    hibernate.go:321,hibernate.go:332,test_runner.go:673: 
        Tests run on Cockroach v19.1.4-4-g9a982e9
        Tests run against Hibernate 5.4.4
        9527 Total Tests Run
        9175 tests passed
        352 tests failed
        3 tests passed unexpectedly
        29 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_1) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/bd27eb358f558bb7598945318240335ebcfcdf13

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1447014&tab=buildLog

The test failed on branch=provisional_201908202216_v19.2.0-beta.20190826, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190821-1447014/hibernate/run_1
    hibernate.go:321,hibernate.go:332,test_runner.go:673: 
        Tests run on Cockroach v19.2.0-alpha.20190606-1654-gbd27eb3
        Tests run against Hibernate 5.4.4
        9527 Total Tests Run
        9201 tests passed
        326 tests failed
        5 tests passed unexpectedly
        17 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_2) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/93860e69f96aa3a86bd8bb42f310fb2629d53f39

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1447036&tab=buildLog

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190821-1447036/hibernate/run_1
    test_runner.go:688: test timed out (10h0m0s)

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/7ca0a86b8595c097fd8f27581b1509c47f17e8a3

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1450654&tab=buildLog

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190823-1450654/hibernate/run_1
    hibernate.go:321,hibernate.go:332,test_runner.go:673: 
        Tests run on Cockroach v19.2.0-alpha.20190606-1701-g7ca0a86
        Tests run against Hibernate 5.4.4
        9487 Total Tests Run
        8882 tests passed
        605 tests failed
        4 tests passed unexpectedly
        295 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_2) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/40f8f0eb00f4b3bf5bac11fb5ae132e33a492713

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1452154&tab=buildLog

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190824-1452154/hibernate/run_1
    hibernate.go:321,hibernate.go:332,test_runner.go:673: 
        Tests run on Cockroach v19.2.0-alpha.20190606-1728-g40f8f0e
        Tests run against Hibernate 5.4.4
        9492 Total Tests Run
        8987 tests passed
        505 tests failed
        5 tests passed unexpectedly
        196 tests failed unexpectedly
        0 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_2) is available in the artifacts' hibernate log

[cockroach-teamcity]

SHA: https://github.com/cockroachdb/cockroach/commits/497167b1c596eda2b70bed91c51ebf39b4356c33

Parameters:

To repro, try:

# Don't forget to check out a clean suitable branch and experiment with the
# stress invocation until the desired results present themselves. For example,
# using stress instead of stressrace and passing the '-p' stressflag which
# controls concurrency.
./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh
cd ~/go/src/github.com/cockroachdb/cockroach && \
stdbuf -oL -eL \
make stressrace TESTS=hibernate PKG=roachtest TESTTIMEOUT=5m STRESSFLAGS='-maxtime 20m -timeout 10m' 2>&1 | tee /tmp/stress.log

Failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=1453099&tab=buildLog

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/20190825-1453099/hibernate/run_1
    hibernate.go:321,hibernate.go:332,test_runner.go:673: 
        Tests run on Cockroach v19.2.0-alpha.20190606-1734-g497167b
        Tests run against Hibernate 5.4.4
        8898 Total Tests Run
        8329 tests passed
        569 tests failed
        5 tests passed unexpectedly
        309 tests failed unexpectedly
        49 tests expected failed, but not run
        For a full summary look at the hibernate artifacts 

        An updated blacklist (hibernateBlackList19_2) is available in the artifacts' hibernate log