search

pa11y / pa11y-ci

Pa11y CI is a CI-centric accessibility test runner, built using Pa11y
https://pa11y.org
GNU Lesser General Public License v3.0
520 stars 63 forks source link

Show errors even when under threshold #70

Closed pinglinh closed 11 months ago

pinglinh commented 5 years ago

I noticed that when errors shown are under the threshold specified, the errors no longer show. Is it possible to make an enhancement to this to add the option of showing these errors anyways? I think it will be good in terms of trying to improve and get rid of these errors before pushing the code to build in our CI pipelines.

42tte commented 4 years ago

Duplicate of #54

dapenguin commented 4 years ago

Had a similar issue last week, but realised it was down to how I was configuring pa11y. I was setting threshold inside my .pa11yci.json file, and so when the errors were below that threshold it wouldn't output what the errors were. However, if I removed the threshold from there and used the -T flag in the CLI, that would still report the errors even if they fell below the threshold.

Don't know if that helps at all.

I-Valchev commented 3 years ago

@42tte hey, I'm bumping into the same issue. It appears the -T or --threshold don't work at all?

I-Valchev commented 3 years ago

hi @42tte ,

You asked if I would try this: pa11y-ci --threshold 83 --config tests/a11y/pa11yci.json

But unfortunately it doesn't work. I much prefer an option in the command line rather than the config, because that way I can use the config for CI (where I don't want it to fail for the number of existing issues, but I'd like it to fail if new ones come up) wheres I want to use that same config locally to look at all the issues (so no threshold).

Is that not possible at the moment? Thanks!

danyalaytekin commented 11 months ago

Thanks all, and for the pointer to #54 @42tte. Let's track this there.