x/build/cmd/watchflakes: should report x/ repo breakage caused by Go repo change

dmitshur commented 2 months ago

A part of issue #58819 was was making watchflakes report, rather than skip, cases of consistent test failures.

golang.org/x/exp/typeparams.TestAPIConsistency is currently broken (e.g., https://ci.chromium.org/b/8740322578467634353) and has been consistently failing on all builders since CL 575455 was submitted about 6 days ago, but it doesn't seem to be reported by watchflakes.

I haven't looked very closely, but the problem might be that the breaking change was in the main Go repo rather than the x/exp repo itself, so looking at the latest x/exp commit in the https://ci.chromium.org/p/golang/g/x-exp-gotip/console view shows it as green, and there haven't been newer commits that landed in x/exp. Many failing commits are visible in the "by Go commit" view at https://ci.chromium.org/p/golang/g/x-exp-gotip-by-go/console, though.

Perhaps there's more to do for watchflakes to report consistent failures when they happen due to a change in the main Go repo. CC @cherrymui, @cagedmantis.

gabyhelp commented 2 months ago

Related Issues and Documentation

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

cherrymui commented 2 months ago

Yeah, "subrepo by go commit" is one thing I want to get on but haven't. There is a TODO https://cs.opensource.google/go/x/build/+/master:cmd/watchflakes/main.go;l=29

golang / go

x/build/cmd/watchflakes: should report x/ repo breakage caused by Go repo change #68753