golang / go

The Go programming language
https://go.dev
BSD 3-Clause "New" or "Revised" License
124.33k stars 17.7k forks source link

cmd/internal/testdir: Test/rangegen.go failures #67556

Open gopherbot opened 6 months ago

gopherbot commented 6 months ago
#!watchflakes
default <- pkg == "cmd/internal/testdir" && test == "Test/rangegen.go"

Issue created automatically to collect these failures.

Example (log):

=== RUN   Test/rangegen.go
=== PAUSE Test/rangegen.go
=== CONT  Test/rangegen.go
    testdir_test.go:145: exit status 1
        command-line-arguments: /home/swarming/.swarming/w/ir/x/w/goroot/pkg/tool/linux_ppc64/compile: signal: killed

--- FAIL: Test/rangegen.go (54.26s)

watchflakes

gopherbot commented 6 months ago

Found new dashboard test flakes for:

#!watchflakes
default <- pkg == "cmd/internal/testdir" && test == "Test/rangegen.go"
2024-04-09 04:07 gotip-linux-ppc64_power8 go@9f3f4c64 cmd/internal/testdir.Test/rangegen.go (log) === RUN Test/rangegen.go === PAUSE Test/rangegen.go === CONT Test/rangegen.go testdir_test.go:145: exit status 1 command-line-arguments: /home/swarming/.swarming/w/ir/x/w/goroot/pkg/tool/linux_ppc64/compile: signal: killed --- FAIL: Test/rangegen.go (54.26s)

watchflakes

pmur commented 6 months ago

The most recent ppc64le power9 failures indicate OOM. The ppc64le containers are limited to 10GB today (including tmpfs for LUCI stuff). The rangegen test is probably the most memory intensive test.

Memory cgroup out of memory: Killed process 2631390 (compile) total-vm:3672064kB, anon-rss:2462528kB, file-rss:128kB, shmem-rss:16704kB, UID:2000 pgtables:350kB oom_score_adj:0

gopherbot commented 4 months ago

Found new dashboard test flakes for:

#!watchflakes
default <- pkg == "cmd/internal/testdir" && test == "Test/rangegen.go"
2024-06-04 15:28 gotip-linux-ppc64le_power9 go@535f81e7 cmd/internal/testdir.Test/rangegen.go (log) === RUN Test/rangegen.go === PAUSE Test/rangegen.go === CONT Test/rangegen.go testdir_test.go:145: exit status 1 command-line-arguments: /home/swarming/.swarming/w/ir/x/w/goroot/pkg/tool/linux_ppc64le/compile: signal: killed --- FAIL: Test/rangegen.go (75.88s)
2024-06-04 16:19 gotip-linux-ppc64le_power9 go@b5a86178 cmd/internal/testdir.Test/rangegen.go (log) === RUN Test/rangegen.go === PAUSE Test/rangegen.go === CONT Test/rangegen.go testdir_test.go:145: exit status 1 command-line-arguments: /home/swarming/.swarming/w/ir/x/w/goroot/pkg/tool/linux_ppc64le/compile: signal: killed --- FAIL: Test/rangegen.go (69.98s)
2024-06-05 19:56 gotip-linux-ppc64le_power9 go@e53d10dc cmd/internal/testdir.Test/rangegen.go (log) === RUN Test/rangegen.go === PAUSE Test/rangegen.go === CONT Test/rangegen.go testdir_test.go:145: exit status 1 command-line-arguments: /home/swarming/.swarming/w/ir/x/w/goroot/pkg/tool/linux_ppc64le/compile: signal: killed --- FAIL: Test/rangegen.go (67.58s)
2024-06-21 12:36 gotip-linux-ppc64le_power9 go@20b79fd5 cmd/internal/testdir.Test/rangegen.go (log) === RUN Test/rangegen.go === PAUSE Test/rangegen.go === CONT Test/rangegen.go testdir_test.go:145: exit status 1 command-line-arguments: /home/swarming/.swarming/w/ir/x/w/goroot/pkg/tool/linux_ppc64le/compile: signal: killed --- FAIL: Test/rangegen.go (58.92s)

watchflakes

gopherbot commented 4 months ago

Found new dashboard test flakes for:

#!watchflakes
default <- pkg == "cmd/internal/testdir" && test == "Test/rangegen.go"
2024-07-17 15:08 gotip-linux-ppc64le_power9 go@fc51e502 cmd/internal/testdir.Test/rangegen.go (log) === RUN Test/rangegen.go === PAUSE Test/rangegen.go === CONT Test/rangegen.go testdir_test.go:145: exit status 1 command-line-arguments: /home/swarming/.swarming/w/ir/x/w/goroot/pkg/tool/linux_ppc64le/compile: signal: killed --- FAIL: Test/rangegen.go (65.10s)

watchflakes

gopherbot commented 4 months ago

Found new dashboard test flakes for:

#!watchflakes
default <- pkg == "cmd/internal/testdir" && test == "Test/rangegen.go"
2024-07-22 15:45 gotip-linux-ppc64le_power9 go@f0de94ff cmd/internal/testdir.Test/rangegen.go (log) === RUN Test/rangegen.go === PAUSE Test/rangegen.go === CONT Test/rangegen.go testdir_test.go:145: exit status 1 command-line-arguments: /home/swarming/.swarming/w/ir/x/w/goroot/pkg/tool/linux_ppc64le/compile: signal: killed --- FAIL: Test/rangegen.go (65.94s)

watchflakes

gopherbot commented 4 months ago

Found new dashboard test flakes for:

#!watchflakes
default <- pkg == "cmd/internal/testdir" && test == "Test/rangegen.go"
2024-07-23 17:16 gotip-linux-ppc64le_power9 go@fe87b586 cmd/internal/testdir.Test/rangegen.go (log) === RUN Test/rangegen.go === PAUSE Test/rangegen.go === CONT Test/rangegen.go testdir_test.go:145: exit status 1 command-line-arguments: /home/swarming/.swarming/w/ir/x/w/goroot/pkg/tool/linux_ppc64le/compile: signal: killed --- FAIL: Test/rangegen.go (66.05s)

watchflakes

gopherbot commented 4 months ago

Found new dashboard test flakes for:

#!watchflakes
default <- pkg == "cmd/internal/testdir" && test == "Test/rangegen.go"
2024-07-25 14:38 gotip-linux-ppc64le_power9 go@e76353d5 cmd/internal/testdir.Test/rangegen.go (log) === RUN Test/rangegen.go === PAUSE Test/rangegen.go === CONT Test/rangegen.go testdir_test.go:145: exit status 1 command-line-arguments: /home/swarming/.swarming/w/ir/x/w/goroot/pkg/tool/linux_ppc64le/compile: signal: killed --- FAIL: Test/rangegen.go (63.98s)

watchflakes

gopherbot commented 4 months ago

Found new dashboard test flakes for:

#!watchflakes
default <- pkg == "cmd/internal/testdir" && test == "Test/rangegen.go"
2024-07-24 18:55 go1.23-linux-ppc64le_power9 release-branch.go1.23@559c7759 cmd/internal/testdir.Test/rangegen.go (log) === RUN Test/rangegen.go === PAUSE Test/rangegen.go === CONT Test/rangegen.go testdir_test.go:145: exit status 1 command-line-arguments: /home/swarming/.swarming/w/ir/x/w/goroot/pkg/tool/linux_ppc64le/compile: signal: killed --- FAIL: Test/rangegen.go (73.00s)

watchflakes

gopherbot commented 4 months ago

Found new dashboard test flakes for:

#!watchflakes
default <- pkg == "cmd/internal/testdir" && test == "Test/rangegen.go"
2024-07-30 17:16 gotip-linux-ppc64le_power9 go@c7227bc7 cmd/internal/testdir.Test/rangegen.go (log) === RUN Test/rangegen.go === PAUSE Test/rangegen.go === CONT Test/rangegen.go testdir_test.go:145: exit status 1 command-line-arguments: /home/swarming/.swarming/w/ir/x/w/goroot/pkg/tool/linux_ppc64le/compile: signal: killed --- FAIL: Test/rangegen.go (57.99s)

watchflakes

gopherbot commented 4 months ago

Found new dashboard test flakes for:

#!watchflakes
default <- pkg == "cmd/internal/testdir" && test == "Test/rangegen.go"
2024-07-31 14:26 gotip-linux-ppc64le_power9 go@eea2e929 cmd/internal/testdir.Test/rangegen.go (log) === RUN Test/rangegen.go === PAUSE Test/rangegen.go === CONT Test/rangegen.go testdir_test.go:145: exit status 1 command-line-arguments: /home/swarming/.swarming/w/ir/x/w/goroot/pkg/tool/linux_ppc64le/compile: signal: killed --- FAIL: Test/rangegen.go (73.02s)

watchflakes

gopherbot commented 4 months ago

Found new dashboard test flakes for:

#!watchflakes
default <- pkg == "cmd/internal/testdir" && test == "Test/rangegen.go"
2024-07-31 21:23 gotip-linux-ppc64le_power9 go@31e692be cmd/internal/testdir.Test/rangegen.go (log) === RUN Test/rangegen.go === PAUSE Test/rangegen.go === CONT Test/rangegen.go testdir_test.go:145: exit status 1 command-line-arguments: /home/swarming/.swarming/w/ir/x/w/goroot/pkg/tool/linux_ppc64le/compile: signal: killed --- FAIL: Test/rangegen.go (73.01s)

watchflakes

gopherbot commented 4 months ago

Found new dashboard test flakes for:

#!watchflakes
default <- pkg == "cmd/internal/testdir" && test == "Test/rangegen.go"
2024-08-01 02:53 gotip-linux-ppc64le_power9 go@bd85a3b1 cmd/internal/testdir.Test/rangegen.go (log) === RUN Test/rangegen.go === PAUSE Test/rangegen.go === CONT Test/rangegen.go testdir_test.go:145: exit status 1 command-line-arguments: /home/swarming/.swarming/w/ir/x/w/goroot/pkg/tool/linux_ppc64le/compile: signal: killed --- FAIL: Test/rangegen.go (76.35s)

watchflakes

gopherbot commented 3 months ago

Found new dashboard test flakes for:

#!watchflakes
default <- pkg == "cmd/internal/testdir" && test == "Test/rangegen.go"
2024-08-01 19:13 gotip-linux-ppc64le_power9 go@2caf638e cmd/internal/testdir.Test/rangegen.go (log) === RUN Test/rangegen.go === PAUSE Test/rangegen.go === CONT Test/rangegen.go testdir_test.go:145: exit status 1 command-line-arguments: /home/swarming/.swarming/w/ir/x/w/goroot/pkg/tool/linux_ppc64le/compile: signal: killed --- FAIL: Test/rangegen.go (55.61s)

watchflakes

gopherbot commented 3 months ago

Found new dashboard test flakes for:

#!watchflakes
default <- pkg == "cmd/internal/testdir" && test == "Test/rangegen.go"
2024-08-02 00:29 gotip-linux-ppc64le_power9 go@11dbbaff cmd/internal/testdir.Test/rangegen.go (log) === RUN Test/rangegen.go === PAUSE Test/rangegen.go === CONT Test/rangegen.go testdir_test.go:145: exit status 1 command-line-arguments: /home/swarming/.swarming/w/ir/x/w/goroot/pkg/tool/linux_ppc64le/compile: signal: killed --- FAIL: Test/rangegen.go (53.98s)
2024-08-02 14:14 gotip-linux-ppc64le_power9 go@9abd1144 cmd/internal/testdir.Test/rangegen.go (log) === RUN Test/rangegen.go === PAUSE Test/rangegen.go === CONT Test/rangegen.go testdir_test.go:145: exit status 1 command-line-arguments: /home/swarming/.swarming/w/ir/x/w/goroot/pkg/tool/linux_ppc64le/compile: signal: killed --- FAIL: Test/rangegen.go (63.39s)

watchflakes

pmur commented 3 months ago

I've readjusted the power9 builders to 4 instances, and reallocated the memory (about 3G more to the remaining buildlets). I suspect the underlying hardware is overcommitted, and that is the cause of the spurious slowdowns we see only on P9.

gopherbot commented 3 months ago

Found new dashboard test flakes for:

#!watchflakes
default <- pkg == "cmd/internal/testdir" && test == "Test/rangegen.go"
2024-08-05 13:12 gotip-linux-ppc64le_power9 go@d465aee0 cmd/internal/testdir.Test/rangegen.go (log) === RUN Test/rangegen.go === PAUSE Test/rangegen.go === CONT Test/rangegen.go testdir_test.go:145: exit status 1 command-line-arguments: /home/swarming/.swarming/w/ir/x/w/goroot/pkg/tool/linux_ppc64le/compile: signal: killed --- FAIL: Test/rangegen.go (55.73s)

watchflakes

gopherbot commented 2 months ago

Found new dashboard test flakes for:

#!watchflakes
default <- pkg == "cmd/internal/testdir" && test == "Test/rangegen.go"
2024-09-20 19:09 gotip-windows-arm64 go@6600a871 cmd/internal/testdir.Test/rangegen.go (log) === RUN Test/rangegen.go === PAUSE Test/rangegen.go === CONT Test/rangegen.go testdir_test.go:147: exit status 1 # command-line-arguments fatal error: runtime: cannot allocate memory runtime stack: runtime.throw({0x7ff7cef77d8e?, 0x407c07a000?}) runtime/panic.go:1069 +0x38 fp=0xde829ffc40 sp=0xde829ffc10 pc=0x7ff7ce6404c8 ... cmd/compile/internal/gc.compileFunctions.func5.1(0x0?) cmd/compile/internal/gc/compile.go:189 +0x40 fp=0x40563cffa0 sp=0x40563cff60 pc=0x7ff7cee47250 cmd/compile/internal/gc.compileFunctions.func3.1() cmd/compile/internal/gc/compile.go:171 +0x3c fp=0x40563cffd0 sp=0x40563cffa0 pc=0x7ff7cee475bc runtime.goexit({}) runtime/asm_arm64.s:1223 +0x4 fp=0x40563cffd0 sp=0x40563cffd0 pc=0x7ff7ce647794 created by cmd/compile/internal/gc.compileFunctions.func3 in goroutine 13 cmd/compile/internal/gc/compile.go:170 +0x1e0 --- FAIL: Test/rangegen.go (28.68s)
2024-09-20 20:19 gotip-windows-arm64 go@097b7162 cmd/internal/testdir.Test/rangegen.go (log) === RUN Test/rangegen.go === PAUSE Test/rangegen.go === CONT Test/rangegen.go testdir_test.go:147: exit status 1 # command-line-arguments fatal error: runtime: cannot allocate memory runtime stack: runtime.throw({0x7ff738817d8e?, 0x7ff737e88b40?}) runtime/panic.go:1069 +0x38 fp=0xc74afffa70 sp=0xc74afffa40 pc=0x7ff737ee04c8 ... runtime.chansend1(0x0?, 0x0?) runtime/chan.go:157 +0x18 fp=0x404699ffa0 sp=0x404699ff70 pc=0x7ff737e67d58 cmd/compile/internal/gc.compileFunctions.func3.1() cmd/compile/internal/gc/compile.go:172 +0x48 fp=0x404699ffd0 sp=0x404699ffa0 pc=0x7ff7386e75c8 runtime.goexit({}) runtime/asm_arm64.s:1223 +0x4 fp=0x404699ffd0 sp=0x404699ffd0 pc=0x7ff737ee7794 created by cmd/compile/internal/gc.compileFunctions.func3 in goroutine 18 cmd/compile/internal/gc/compile.go:170 +0x1e0 --- FAIL: Test/rangegen.go (43.82s)

watchflakes

gopherbot commented 1 month ago

Found new dashboard test flakes for:

#!watchflakes
default <- pkg == "cmd/internal/testdir" && test == "Test/rangegen.go"
2024-10-28 21:25 gotip-windows-arm64 go@775837f5 cmd/internal/testdir.Test/rangegen.go (log) === RUN Test/rangegen.go === PAUSE Test/rangegen.go === CONT Test/rangegen.go testdir_test.go:147: exit status 1 # command-line-arguments runtime: VirtualAlloc of 8192 bytes failed with errno=1455 fatal error: out of memory runtime stack: runtime.throw({0x7ff7abe604ea?, 0x408aa58000?}) ... runtime.chansend1(0x0?, 0x400005df80?) runtime/chan.go:157 +0x18 fp=0x40573f3fa0 sp=0x40573f3f70 pc=0x7ff7ab4b7c28 cmd/compile/internal/gc.compileFunctions.func3.1() cmd/compile/internal/gc/compile.go:172 +0x48 fp=0x40573f3fd0 sp=0x40573f3fa0 pc=0x7ff7abd3f7e8 runtime.goexit({}) runtime/asm_arm64.s:1260 +0x4 fp=0x40573f3fd0 sp=0x40573f3fd0 pc=0x7ff7ab537744 created by cmd/compile/internal/gc.compileFunctions.func3 in goroutine 22 cmd/compile/internal/gc/compile.go:170 +0x1e0 --- FAIL: Test/rangegen.go (45.51s)
2024-10-28 21:25 gotip-windows-arm64 go@cd54b9ba cmd/internal/testdir.Test/rangegen.go (log) === RUN Test/rangegen.go === PAUSE Test/rangegen.go === CONT Test/rangegen.go testdir_test.go:147: exit status 1 # command-line-arguments runtime: VirtualAlloc of 8192 bytes failed with errno=1455 fatal error: out of memory runtime: VirtualAlloc of 8192 bytes failed with errno=1455 fatal error: out of memory runtime: VirtualAlloc of 8192 bytes failed with errno=1455 fatal error: out of memory runtime stack: runtime.throw({0x7ff696fa04f5?, 0x408fb0c000?}) runtime/panic.go:1074 +0x38 fp=0x7c95fff600 sp=0x7c95fff5d0 pc=0x7ff696670268 runtime.sysUsedOS(0x408fb0c000, 0x2000) runtime/mem_windows.go:83 +0x178 fp=0x7c95fff660 sp=0x7c95fff600 pc=0x7ff69660a768 runtime.sysUsed(...) runtime/mem.go:77 runtime.(*mheap).allocSpan(0x7ff697768760, 0x1, 0x0, 0x26) ... runtime.sysUsed(...) runtime/mem.go:77 runtime.(*mheap).allocSpan(0x7ff697768760, 0x1, 0x0, 0x3f) runtime/mheap.go:1345 +0x880 fp=0x7c953ff7f0 sp=0x7c953ff6d0 pc=0x7ff696620fe0 runtime.(*mheap).alloc.func1() runtime/mheap.go:962 +0x6c fp=0x7c953ff840 sp=0x7c953ff7f0 pc=0x7ff69662024c runtime.systemstack(0x86) runtime/asm_arm64.s:281 +0x6c fp=0x7c953ff850 sp=0x7c953ff840 pc=0x7ff69667531c --- FAIL: Test/rangegen.go (44.41s)

watchflakes

gopherbot commented 4 days ago

Found new dashboard test flakes for:

#!watchflakes
default <- pkg == "cmd/internal/testdir" && test == "Test/rangegen.go"
2024-11-20 22:41 gotip-windows-arm64 go@dce30a19 cmd/internal/testdir.Test/rangegen.go (log) === RUN Test/rangegen.go === PAUSE Test/rangegen.go === CONT Test/rangegen.go testdir_test.go:147: exit status 1 # command-line-arguments fatal error: runtime: cannot allocate memory runtime stack: runtime.throw({0x7ff6498adca5?, 0x404d630000?}) runtime/panic.go:1099 +0x38 fp=0xa97cdff840 sp=0xa97cdff810 pc=0x7ff648f29ae8 ... cmd/compile/internal/gc.compileFunctions.func5.1(0x0?) cmd/compile/internal/gc/compile.go:189 +0x40 fp=0x4048dbffa0 sp=0x4048dbff60 pc=0x7ff649754610 cmd/compile/internal/gc.compileFunctions.func3.1() cmd/compile/internal/gc/compile.go:171 +0x3c fp=0x4048dbffd0 sp=0x4048dbffa0 pc=0x7ff64975497c runtime.goexit({}) runtime/asm_arm64.s:1260 +0x4 fp=0x4048dbffd0 sp=0x4048dbffd0 pc=0x7ff648f31124 created by cmd/compile/internal/gc.compileFunctions.func3 in goroutine 9 cmd/compile/internal/gc/compile.go:170 +0x1e0 --- FAIL: Test/rangegen.go (25.68s)
2024-11-21 15:06 gotip-windows-arm64 go@0a0a7a56 cmd/internal/testdir.Test/rangegen.go (log) === RUN Test/rangegen.go === PAUSE Test/rangegen.go === CONT Test/rangegen.go testdir_test.go:147: exit status 1 # command-line-arguments runtime: VirtualAlloc of 16384 bytes failed with errno=1455 fatal error: out of memory runtime stack: runtime.throw({0x7ff75eb87e71?, 0x40613c0000?}) ... runtime.chansend1(0x0?, 0x0?) runtime/chan.go:161 +0x18 fp=0x4060a6dfa0 sp=0x4060a6df70 pc=0x7ff75e1a7d78 cmd/compile/internal/gc.compileFunctions.func3.1() cmd/compile/internal/gc/compile.go:172 +0x48 fp=0x4060a6dfd0 sp=0x4060a6dfa0 pc=0x7ff75ea44a38 runtime.goexit({}) runtime/asm_arm64.s:1260 +0x4 fp=0x4060a6dfd0 sp=0x4060a6dfd0 pc=0x7ff75e221114 created by cmd/compile/internal/gc.compileFunctions.func3 in goroutine 9 cmd/compile/internal/gc/compile.go:170 +0x1e0 --- FAIL: Test/rangegen.go (29.50s)
2024-11-21 15:20 gotip-windows-arm64 go@57147256 cmd/internal/testdir.Test/rangegen.go (log) === RUN Test/rangegen.go === PAUSE Test/rangegen.go === CONT Test/rangegen.go testdir_test.go:147: exit status 1 # command-line-arguments runtime: VirtualAlloc of 40960 bytes failed with errno=1455 fatal error: out of memory runtime stack: runtime.throw({0x7ff716cc7e71?, 0x406049a000?}) ... cmd/compile/internal/gc.compileFunctions.func5.1(0x0?) cmd/compile/internal/gc/compile.go:189 +0x40 fp=0x4060343fa0 sp=0x4060343f60 pc=0x7ff716b846c0 cmd/compile/internal/gc.compileFunctions.func3.1() cmd/compile/internal/gc/compile.go:171 +0x3c fp=0x4060343fd0 sp=0x4060343fa0 pc=0x7ff716b84a2c runtime.goexit({}) runtime/asm_arm64.s:1260 +0x4 fp=0x4060343fd0 sp=0x4060343fd0 pc=0x7ff716361114 created by cmd/compile/internal/gc.compileFunctions.func3 in goroutine 18 cmd/compile/internal/gc/compile.go:170 +0x1e0 --- FAIL: Test/rangegen.go (30.45s)
2024-11-21 15:44 gotip-windows-arm64 go@a5c1b1c4 cmd/internal/testdir.Test/rangegen.go (log) === RUN Test/rangegen.go === PAUSE Test/rangegen.go === CONT Test/rangegen.go testdir_test.go:147: exit status 1 # command-line-arguments fatal error: runtime: cannot allocate memory runtime stack: runtime.throw({0x7ff79df7dd4f?, 0x407a5ac000?}) runtime/panic.go:1099 +0x38 fp=0xf5b3dff560 sp=0xf5b3dff530 pc=0x7ff79d5f9ad8 ... cmd/compile/internal/gc.compileFunctions.func5.1(0x40003f1040?) cmd/compile/internal/gc/compile.go:189 +0x40 fp=0x4072e9bfa0 sp=0x4072e9bf60 pc=0x7ff79de246c0 cmd/compile/internal/gc.compileFunctions.func3.1() cmd/compile/internal/gc/compile.go:171 +0x3c fp=0x4072e9bfd0 sp=0x4072e9bfa0 pc=0x7ff79de24a2c runtime.goexit({}) runtime/asm_arm64.s:1260 +0x4 fp=0x4072e9bfd0 sp=0x4072e9bfd0 pc=0x7ff79d601114 created by cmd/compile/internal/gc.compileFunctions.func3 in goroutine 9 cmd/compile/internal/gc/compile.go:170 +0x1e0 --- FAIL: Test/rangegen.go (34.65s)

watchflakes