search

WordPress / phpunit-test-reporter

GNU General Public License v3.0
29 stars 21 forks source link

Alert hosts if tests haven't been reported for a period of time. #61

Open getsource opened 4 years ago

getsource commented 4 years ago

Hosts are contacted if tests fail when they are passing on core's TravisCI, but not when they haven't reported in some time.

There's a section on the test results page for "Registered, but no reports in >25 Revisions", and hosts may become listed there without noticing their testing infrastructure has stalled.

Let's see if we can keep that from happening as frequently.

getsource commented 4 years ago

A note that @pfefferle recommended including a link to the hosting community Slack channel, making sure that it mentions that the team is available to help + support them.

getsource commented 4 years ago

This ticket is a part of getting and keeping hosts connected re: #87

Just finished chatting with @pfefferle about this idea, and wanted to make sure it was tracked here.

One way we might be able to do this is to alert based on count of commits away from trunk, triggered by Core's reports from Travis CI.

This would avoid cron and usermeta requirements, and continue to base the metric on commits.

I think we could either do a check and email all inactive reporters each X commits to core (like % 50, or % 100), or individually, based on how many commits the reporter is behind.

The latter is better, I think -- but whichever is lower impact would probably serve the purpose.