The global options concurrency and useIncognitoBrowserContext default to values that are not great for a CI environment:
concurrency is set by default at 2. Chrome is quite resource hungry, and certain Continuous Integration (CI) environments (specially in the free-tier) tend to be quite constraint, so this may cause Chrome to fail to run properly. A better default would be to set concurrency to 1 and allow people to increase it if, and only if, they know that their CI environment will be able to cope with it.
useIncognitoBrowserContext is set by default to false. This will cause pa11y to re-use existing cookies (inconsistently, as concurrency is set to 2) and potentially cause lots of problems when dealing with authentication, or cause pa11y to produce inconsistent results. We should consider defaulting to having completely independent browser contexts for each tab so tests are as reproducible as possible.
Definitely a breaking change, so scheduling for milestone 3.0.
The global options
concurrencyanduseIncognitoBrowserContextdefault to values that are not great for a CI environment:concurrencyis set by default at2. Chrome is quite resource hungry, and certain Continuous Integration (CI) environments (specially in the free-tier) tend to be quite constraint, so this may cause Chrome to fail to run properly. A better default would be to setconcurrencyto 1 and allow people to increase it if, and only if, they know that their CI environment will be able to cope with it.useIncognitoBrowserContextis set by default tofalse. This will cause pa11y to re-use existing cookies (inconsistently, asconcurrencyis set to 2) and potentially cause lots of problems when dealing with authentication, or cause pa11y to produce inconsistent results. We should consider defaulting to having completely independent browser contexts for each tab so tests are as reproducible as possible.Definitely a breaking change, so scheduling for milestone 3.0.
Environment: