Closed marklaatikainen closed 2 years ago
@marklaatikainen - could you please update this issue to provide more details in the "To Reproduce" section and "Expected behavior" section? It looks like you provided the output of the GitHub Action which looks correct.
Please provide your configuration or some similar example configuration. I would expect there to be no scores in the output and an error message per this recent PR. Do you not see an error message logged in the output?
Is the problem that you are using foo-software/lighthouse-check-status-action
and the workflow doesn't fail when you would expect it to? If so, you would definitely need to at least specify the relevant configuration for that (it's actually a different GitHub Action altogether... but it is related to this, so we can keep this open).
Regardless, I will need a minimal example and more information to understand the issue better.
Configuration looks like this. There are separate jobs for desktop and mobile, with different minimum scores.
It gives following output and passes both steps:
I see... thank you @marklaatikainen. As I suspected - this GitHub Action is actually working as expected. The issue resides in lighthouse-check-status-action, which is a different repo altogether.
Let's keep this open here since the other repo isn't well known. I'm busy today and tomorrow, but should be able to get to the bottom of it on Friday.
@marklaatikainen thanks again for opening this issue. It has been resolved and released in foo-software/lighthouse-check-action@9.0.0.
We now "fast fail" a job when a runtime error occurs. Example
Closing this, but feel free to comment or re-open if needed.
Thank you for quick fix 👍 Seems to be working now. @adamhenson
👍
Describe the bug Step passes with runtime error
To Reproduce Run check against url that loads slowly
Expected behavior Step should fail if page could not load
Additional context Our webstore preview link takes long time to load on first load after it was built. Step passes still with SUCCESS when it has
runtimeError
Actual result lighthouse-check: Scores passed minimum requirement ✅