is there a way to create aggregated report ?

[mahesh-nihr]

Hi thanks for awesome package. I am able to create reports for individual pages but just wondering is there a way to create aggregated report for results from multiple pages ?

Thanks

[lpelypenko]

Hi @mahesh-nihr, thank you very much for your request. Short answer - no :)

I was waiting for specific need to gather requirements for this feature and contribution from the user to come up with joined requirements.

In my team we had need to aggregate results but we wanted to deduplicate raw axe violation and use deduplicated violations (not other parts) to create one html file. Currently, I approach this flow in following way (testCafe example):

1. Run axe check (with @testcafe-community/axe library) and store raw violations in constant, e.g constant name homepage
2. In the same test, run axe check for other page and store in constant name, e.g loginPage
3. ... repeat step 1-2 for as many pages
4. Deduplicate raw AXE violations using my simple violations deduplication library (currently not public) to get one raw results without duplication of violations
5. Create html report from deduplicated violations only

I assumed that this flow may not work for everyone and complicated, so I haven't published axe violation deduplication lib.

I would really appreciate your input on possible implementations of your request. I am not sure what would be your flow and which framework you use to run axe check on the page, but I can propose following options for future implementation: Option 1: create a new function(e.g createAggregatedHtmlReport) in axe-html-reporter to create a report from multiple raw axe results and show result from every page with tabs. Function call can look like this:

const { error1, results1 } = await runAxe(axeContext, axeOptions);
await t.expect(error1).notOk(`axe check failed with an error: ${error.message}`);
const { error2, results2 } = await runAxe(axeContext, axeOptions);
await t.expect(error2).notOk(`axe check failed with an error: ${error.message}`);
createAggregatedHtmlReport([{
        results: results1
        options: {
            tabName: 'First page results',
        },
    },
    {
        results: results2
        options: {
            tabName: 'Second page results',
        },
    }
]);

Option 2: create new function(same name, e.g createAggregatedHtmlReport) that will read individual html files from specific directory, read all options from them. Function call can look like this:

// somewhere in your code
    const { error, results } = await runAxe(axeContext, axeOptions);
    await t.expect(error).notOk(`axe check failed with an error: ${error.message}`);
    // creates html report with the default file name `accessibilityReport.html`
    createHtmlReport({
        results,
        options: {
            projectKey: 'Page One results', 
            outputDir: 'axe-reports',
            reportFileName: 'pageOneReport.html',
        },
    });

// somewhere else in your code
    const { error, results } = await runAxe(axeContext, axeOptions);
    await t.expect(error).notOk(`axe check failed with an error: ${error.message}`);
    // creates html report with the default file name `accessibilityReport.html`
    createHtmlReport({
        results,
        options: {
            projectKey: 'Page Two results', 
            outputDir: 'axe-reports',
            reportFileName: 'pageTwoReport.html',
        },
    });

// somewhere where you want to aggregate results from 'axe-reports' directory
createAggregatedHtmlReportFromDir('axe-reports'); 
// output: creates file `aggregatedAxeRestuls.thml` from two files or more from directory 'axe-reports'

Option 3: publish in npm my violations deduplication library that accepts array of violations from different runs and create one array where violations are joined by violation id. Advantages of Option 3: it is already ready, just not published Disadvantages of Option 3: it deduplicates only violations and does not work for incomplete or inapplicable

My personal preference is Option 2 because you can run it without relying on raw results to be available during it's run time.

[harir5]

Hi @lpelypenko - i am also working on an aggregate report. what i have done is as follows :

//on playwright, with aXe-playwright :
const results1 = await getViolations(page);
const results2 = await getViolations(page);
const results = [...results1, ..results2]; //this particular results object is not deduplicated.

createHtmlReport({
      results: { violations: results },
      options: {
        projectKey: "AxE Test",
        customSummary:
          summary,
        outputDir: "./axe-reports",
        reportFileName: "a11yReport.html",
      },
    });
  });

I would like to know if your library for deduplication is available publicly.

P.s - really amazing work on this reporter. it is really wonderful that an extensive, readable report is generated.