search

golang / go

The Go programming language
https://go.dev
BSD 3-Clause "New" or "Revised" License
123.68k stars 17.62k forks source link

x/pkgsite/tests/screentest: module-with-one-vuln is failing (and prone to fail anew after being updated) #68467

Closed dmitshur closed 3 months ago

dmitshur commented 3 months ago

The screentest "module-with-one-vuln" is currently failing at tip, as seen in the most recent trybot occurrence in CL 598524.

This test case was very recently fixed in CL 598035 (CC @tatianab), reflecting that it's no longer the latest published version of the module, but it seems it broke quite quickly since then. In the updated image, the screenshot's badge said "Main | passing" which matches https://pkg.go.dev/github.com/stacklok/minder@v0.0.47 now but might not whenever that project's main branch has a problem. The coverage percentage badge changed from 53% to 54% and members badge text went 134 → 133, contributing to the overall visual diff.

The main source of flakiness in this test case seems that the badges in the README change whenever the tip version of the minder module changes, causing visual diffs.

CC @golang/pkgsite.

gopherbot commented 3 months ago

Change https://go.dev/cl/598532 mentions this issue: tests/screentest: delete flaky screentest

dmitshur commented 3 months ago

CL 598532 deleted this test case. Thanks.