servo / highfive

Github hooks to provide an encouraging atmosphere for new contributors
Mozilla Public License 2.0
254 stars 58 forks source link

Make highfive post the test failures from homu #86

Closed wafflespeanut closed 8 years ago

wafflespeanut commented 8 years ago

This makes use of homu's JSON API to get the test output from stdout and comment on the PR in the event of a test failure.

@jdm I didn't make use of the error summary log, because there seem to be some discrepancies between stdout and the log. For instance, the log doesn't have the stacktraces sometimes[1].

We need the stacktraces because: 1) we need the whole log related to the failure (so that we don't have to visit the stdio page again 2) we can make highfive more intelligent (in the near future) by posting "retry" messages to @bors-servo referencing the relevant intermittent issues or opening a new issue (which will save more time for us!), and for opening new intermittent-related issues, we'll be needing the stacktrace.

[1]: See this build's stdout and the corresponding error summary log

highfive commented 8 years ago

Heads up! This PR modifies the following files:

wafflespeanut commented 8 years ago

Maybe we should rely on stdout until we get stacktraces in the log summary.

jdm commented 8 years ago

Pretty slick! It would be nice to isolate the code that's in charge of figuring out the failures into a separate function, though.

wafflespeanut commented 8 years ago

Oh yeah, okay. I'll go ahead and update the tests then :)

jdm commented 8 years ago

It's confusing that the test changes don't appear to cause a new comment to be added...

jdm commented 8 years ago

@bors-servo: r+

bors-servo commented 8 years ago

:pushpin: Commit 8df76cd has been approved by jdm

bors-servo commented 8 years ago

:hourglass: Testing commit 8df76cd with merge cb064cb...

jdm commented 8 years ago

@bors-servo: retry

bors-servo commented 8 years ago

:hourglass: Testing commit 8df76cd with merge 8c6c741...