lighthouse bot succeeds even when minimum required threshold is not met

GoogleChromeLabs / lighthousebot

Run Lighthouse in CI, as a web service, using Docker. Pass/Fail GH pull requests.

Apache License 2.0

2.24k stars 127 forks source link

lighthouse bot succeeds even when minimum required threshold is not met #62

Closed StudioSpindle closed 5 years ago

StudioSpindle commented 5 years ago

See the output table on: https://github.com/StudioSpindle/spindle.wtf/pull/7#issuecomment-492333770

The settings in Travis are set like this: https://github.com/StudioSpindle/spindle.wtf/blob/develop/.travis.yml

Is this a bug or is it a setting that is wrong?

erezrokah commented 5 years ago

@StudioSpindle I had a similar issue. I found out it takes some time until the lighthousebot account responds to the invitation to collaborate.

You can check this here:

EoinTraynor commented 5 years ago

Until Lighthouse accepts the invitation to collaborate it does not have permission to fail a check. However, it will post a comment on your PR. Here is the original issue Issue 8

@erezrokah How long did you have to wait for acceptance? I requested collaboration 6 days ago.

erezrokah commented 5 years ago

@EoinTraynor I don't remember exactly. I think a couple of hours