search

openzfs / zfs-buildbot

The OpenZFS Buildbot Configuration
http://build.zfsonlinux.org
BSD 2-Clause "Simplified" License
25 stars 29 forks source link

Is there a known-issues.html with PRs? #239

Open rincebrain opened 2 years ago

rincebrain commented 2 years ago

I see that known-issues.sh knows how to generate the page with PR failures included, but that that option presumably isn't being used here.

Is there/could there be a version of that page added to whatever cron generates it that includes PRs, perhaps with a shorter timescale (e.g. 7/14 days)? I suspect it would make things like the issues behind openzfs/zfs#12663 or #238 flamingly obvious...

behlendorf commented 2 years ago

We could, and we used to. In practice we found that including all of the PR failures wasn't that helpful, the results from the daily builds of the master and release branches were sufficient. Here's an example of the output including PRs from the last 14 days. Note it won't include and test results from the GitHub runners, only what was run by buildbot.

rincebrain commented 2 years ago

Heh, I see. It seems like it might benefit in signal:noise from the "ignore exclusions" logic being made not exclusive to the release branches?

I still think it could be pretty useful as a tool for spotting things, maybe with different runs for 1/7/14d failure rates to spot sudden/recurring excitement? Just a thought.

behlendorf commented 2 years ago

Let's try it out and see. I made a quick change to the script apply the ignore exclusions to the PRs, then setup cron jobs for 1 day, 7 days, 14 days, and 30 days.

rincebrain commented 2 years ago

Neat, thanks for humoring my suggestion.

It seems to be doing something strange, though - e.g. here it's seeing failures from 'tests/funct,79:0ional/cli_root/zpool_expand/zpool_expand_001_pos'. I don't immediately see anywhere in the logfiles that's coming from - my guess would be if it were doing grep --color implicitly somewhere, though 79 doesn't ring any bells for any sequence I recognize...

It's also only for zpool_expand tests, and I don't obviously see something in the logs causing that...,hm.

behlendorf commented 2 years ago

Yes, I've noticed that in the past as well. It's not always limited to the expand tests, I seen it elsewhere. Though I never did determine where it's coming from.