Open thibaudcolas opened 2 years ago
@thibaudcolas Apologies for the delay,
Thank you for reporting this.
There are a few ways we could handle this case:
👍 no problem. I like both approaches, to me what matters most is to get feedback on the fact I used an invalid rule ID as early as possible.
This happened to me again just yesterday, when I forgot th-has-data-cells
was plural. It’s not hard to fix but does take a bit of time to understand what is happening.
Axe-core throws an error when passed an invalid rule to run, so we should surface that error to the user as soon as we get it from axe. We shouldn't try to correct the issue about the invalid rule and run axe without it.
Product: CLI
Expectation:
When I use the CLI with the
-r
option to specify which rules to run, and provide an invalid rule ID, I’d expect the CLI to exit early with a relevant error message.Actual:
When providing an invalid rule ID, the CLI takes much longer to report a failure, and the failure message is a very generic "An error occurred while testing this page.".
Motivation:
Here’s a comparison of three scenarios:
I imagine there is a 20s timeout here. It feels unexpected to me that the CLI (or Axe core itself?) would have to wait for such a timeout when it’s not provided with any valid rules to run. Even if there was no early validation of the rule IDs provided, when the CLI runs, it should fail right away, with either a generic "there are no rules to run" or specific "invalid rule ID".