Open mknyszek opened 1 year ago
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
@golang/s390x, note that the Go porting policy requires at least two developers “making required updates in a timely manner as architecture or operating system requirements change.”
It appears from the lack of activity on this issue and the lack of responses to https://github.com/golang/go/issues/60109#issuecomment-1668009698 that this port is no longer meeting that requirement. (The last reply I see from a port maintainer was from @srinivas-pokala in https://github.com/golang/go/issues/60109#issuecomment-1592483875, two months ago.)
@bcmills due to some other work items related s390x port didn't get bandwidth, we are taking a look and we will come up with some solution for this.
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
@bcmills, How can I reproduce this timeout issue? I am not experiencing any timeout issues on my local machine (s390x) with the master branch go repo's tip. However, these tests are being running on a community cloud vm builder (reverse). How to reproduce them can you provide some details to reproduce? Also, there is a parameter GO_TEST_TIMEOUT_SCALE that scales the test timeout. How can we validate the optimal value of this parameter to avoid timeout issues?
@bcmills, I have tried running the build/tests using (./all.bash) on community cloud machine(with tip of master), I am not able to see any timeout issues. Can you provide some details on reproducing this issue.
@srinivas-pokala, this is almost certainly a configuration issue with the builder. I would not expect it to reproduce on a local machine if that machine is not as heavily loaded as the CI builder. 😅
I expect that it can be resolved by changing the configuration of the builder, by running fewer tests in parallel or giving the tests a longer timeout. You may also want to check it for related performance issues (such as hitting swap while running tests).
Generally GO_TEST_TIMEOUT_SCALE
should be set such that the actual timeouts are about an order of magnitude longer than the longest test. The longest one I see in a recent run (arbitrarily chosen from https://build.golang.org/) is:
ok cmd/cgo/internal/testshared 93.248s
10x that would be 15.5 minutes, or about 5x longer than the default 3m timeout set by cmd/dist
, which suggests GO_TEST_TIMEOUT_SCALE=5
.
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Found new dashboard test flakes for:
#!watchflakes
post <- goarch == "s390x" && (`test timed out` || `panic: finalizer started .* ago .* and ran \d+ iterations, but did not return` || `failed to start: context deadline exceeded` || `WaitDelay expired before I/O complete`)
Change https://go.dev/cl/523895 mentions this issue: dashboard: set GO_TEST_TIMEOUT_SCALE for linux-s390x-ibm builder
Broken out from #57636.
CC @golang/s390x