search

cockroachdb / pebble

RocksDB/LevelDB inspired key-value database in Go
BSD 3-Clause "New" or "Revised" License
4.6k stars 424 forks source link

: failed #3686

Closed cockroach-teamcity closed 5 days ago

cockroach-teamcity commented 1 week ago

. failed with artifacts on refs/heads/master @ ff83363e89ce:

[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 9227s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 9287s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 9347s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 9407s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 9467s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 9527s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 9588s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 9648s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 9708s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 9768s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 9828s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 9888s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 9948s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 10008s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 10068s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 10128s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 10188s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 10248s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 10308s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 10368s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 10428s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 10488s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 10548s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 10608s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 10668s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 10728s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 10788s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 10848s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 10908s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 10968s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 11028s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 11088s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 11148s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 11208s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 11268s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 11328s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 11388s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 11449s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 11509s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 11569s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 11629s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 11689s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 11749s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 11809s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 11869s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 11929s processwrapper-sandbox
[821 / 822] Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 11989s processwrapper-sandbox
FAIL: @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test (see /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/execroot/com_github_cockroachdb_cockroach/bazel-out/k8-fastbuild/testlogs/external/com_github_cockroachdb_pebble/internal/metamorphic/crossversion/crossversion_test/test.log)
[821 / 822] 1 / 1 tests, 1 failed; Testing @com_github_cockroachdb_pebble//internal/metamorphic/crossversion:crossversion_test; 12021s processwrapper-sandbox
INFO: From Testing @com_github_cockroachdb_pebble//internal/me
Help

See also: [How To Investigate a Go Test Failure \(internal\)](https://cockroachlabs.atlassian.net/l/c/HgfXfJgM)

/cc @cockroachdb/test-eng

This test on roachdash | Improve this report!

RaduBerinde commented 5 days ago

This is the same with #3635; we increased the per-run timeout but this happened again. #3681 tracks an operation timeout - the underlying cause could be the same.

[07:31:54 ](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_NightlyPebbleMetamorphicCrossversion/15792224?buildTab=log&focusLine=6466&logView=flowAware&linesState=6466)  ERROR: signal: killed
[07:31:54 ](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_NightlyPebbleMetamorphicCrossversion/15792224?buildTab=log&focusLine=6467&logView=flowAware&linesState=6467)  
[07:31:54 ](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_NightlyPebbleMetamorphicCrossversion/15792224?buildTab=log&focusLine=6468&logView=flowAware&linesState=6468)  58 runs completed, 1 failures, over 3h20m22s

And

Failed to publish artifacts: java.lang.OutOfMemoryError: Java heap space
RaduBerinde commented 5 days ago

Oh, https://github.com/cockroachdb/cockroach/pull/125892 increased the timeout but not for this particular build.

Timeout here happened at 20m:

  56 runs so far, 0 failures, over 3h0m20s
  57 runs so far, 0 failures, over 3h0m25s
...
  57 runs so far, 0 failures, over 3h20m20s