search

treosh / lighthouse-ci-action

Audit URLs using Lighthouse and test performance with Lighthouse CI.
MIT License
1.17k stars 82 forks source link

default of `runs: 1` overrides a config's numberOfRuns #48

Closed paulirish closed 4 years ago

paulirish commented 4 years ago

looked into this and the CLI arg is set regardless.. and the CLI will prioritize its args above anything in the config. (which makes sense IMO)

So we shouldn't set the CLI numberOfRuns if it was just falling back to the actions' default of 1.

There's an alternative though.. Instead, the action could just default to the default of the CLI (which is 3 runs). If we did that, then you could just delete some code and this problem would work itself out. ;)

alekseykulikov commented 4 years ago

So we shouldn't set the CLI numberOfRuns if it was just falling back to the actions' default of 1.

I agree with this strategy for now.

the action could just default to the default of the CLI (which is 3 runs)

I don't think that the 3-runs is an optimal default. It may results in a 3x bill with no significant value. 1-run works for static checks, like resource budgets or lighthouse scores. 3-runs makes sense when user tests performance or collect analytics with LHCI Server.

Maybe we could fallback to LHCI default (breaking change for LH v6 release) and show a warning when 3-runs used without an extra value.

alekseykulikov commented 4 years ago

fixed in treosh/lighthouse-ci-action@v3