The artifact refresh in https://github.com/GoogleChrome/lighthouse/pull/15962 revealed that we were not always handling the hidden group correctly. Before, if an audit had group hidden it would only be omitted from the report if it would be in the failed clump. So an audit marked N/A that had group hiddenwould appear in the report (under the N/A section)
This wasn't picked up by unit tests because the unit tests were indifferent to the hidden group and the sample artifacts didn't have any audits with group hidden that failed. With the artifact refresh, there was an audit that failed but was in group hidden which started tripping up the tests with incorrect expectations.
The artifact refresh in https://github.com/GoogleChrome/lighthouse/pull/15962 revealed that we were not always handling the
hiddengroup correctly. Before, if an audit had grouphiddenit would only be omitted from the report if it would be in thefailedclump. So an audit marked N/A that had grouphiddenwould appear in the report (under the N/A section)This wasn't picked up by unit tests because the unit tests were indifferent to the
hiddengroup and the sample artifacts didn't have any audits with grouphiddenthat failed. With the artifact refresh, there was an audit that failed but was in grouphiddenwhich started tripping up the tests with incorrect expectations.This PR splits out the report fix and the associated test expectation corrections from https://github.com/GoogleChrome/lighthouse/pull/15962