Selenium now adds an attribute to each iframe called cd_frame_id which is some unique UUID. This is the first difference, as for runPartial we do not rely on Selenium to collect all of the frames thus no cd_frame_id attribute being added onto each iframe. This causes the equality check between results to fail. An HTML snippet captured by axe:
Doing only one navigation in the WebDriverJS test causes some of the remanence of the legacy run notably the cd_frame_id attribute and causes the aria-hidden-rule to only capture the body. Removing the additional navigation in the Java test causes the test to pass. On the other hand, adding the additional navigation in the Typescript test causes the test to fail.
Follow up that needs to happen:
Identify if having the separate navigations between runs is the correct way for this test to be constructed. If we conclude that it is, we need to follow up on why axe-core is only capturing the body element for legacy and everything inside body for normal run. Second, how we can do true equality checks when selenium adds cd_frame_id onto each iframe this fundamentally breaks our test
Amend the test accordingly from the outcomes of 1
(see below comments for full axe-results generated from each run)
",
"target": [
"#ifr-baz",
"h1"
],
"impact": null,
"any": [
{
"id": "color-contrast",
"impact": "serious",
"message": "Element has sufficient color contrast of 21",
"data": {
"bgColor": "#ffffff",
"contrastRatio": 21,
"expectedContrastRatio": "3:1",
"fgColor": "#000000",
"fontSize": "24.0pt (32px)",
"fontWeight": "bold"
},
"relatedNodes": []
}
],
"all": [],
"none": [],
"failureSummary": null
},
{
"html": "",
"target": [
"#ifr-baz",
"input"
],
"impact": null,
"any": [
{
"id": "color-contrast",
"impact": "serious",
"message": "Element has sufficient color contrast of 21",
"data": {
"bgColor": "#ffffff",
"contrastRatio": 21,
"expectedContrastRatio": "4.5:1",
"fgColor": "#000000",
"fontSize": "10.0pt (13.3333px)",
"fontWeight": "normal"
},
"relatedNodes": []
}
],
"all": [],
"none": [],
"failureSummary": null
}
],
"url": null,
"createdDate": null
},
{
"id": "document-title",
"description": "Ensure each HTML document contains a non-empty element",
"help": "Documents must have <title> element to aid in navigation",
"helpUrl": "https://dequeuniversity.com/rules/axe/4.10/document-title?application=axeAPI",
"impact": "",
"tags": [
"cat.text-alternatives",
"wcag2a",
"wcag242",
"TTv5",
"TT12.a",
"EN-301-549",
"EN-9.2.4.2",
"ACT"
],
"nodes": [
{
"html": "<html lang=\"en\">",
"target": [
"html"
],
"impact": null,
"any": [
{
"id": "doc-has-title",
"impact": "serious",
"message": "Document has a non-empty <title> element",
"data": null,
"relatedNodes": []
}
],
"all": [],
"none": [],
"failureSummary": null
}
],
"url": null,
"createdDate": null
},
{
"id": "empty-heading",
"description": "Ensure headings have discernible text",
"help": "Headings should not be empty",
"helpUrl": "https://dequeuniversity.com/rules/axe/4.10/empty-heading?application=axeAPI",
"impact": "",
"tags": [
"cat.name-role-value",
"best-practice"
],
"nodes": [
{
"html": "<h1>This page has nested frames!</h1>",
"target": [
"h1"
],
"impact": null,
"any": [
{
"id": "has-visible-text",
"impact": "minor",
"message": "Element has text that is visible to screen readers",
"data": null,
"relatedNodes": []
}
],
"all": [],
"none": [],
"failureSummary": null
},
{
"html": "<h1>Foo</h1>",
"target": [
"#ifr-foo",
"h1"
],
"impact": null,
"any": [
{
"id": "has-visible-text",
"impact": "minor",
"message": "Element has text that is visible to screen readers",
"data": null,
"relatedNodes": []
}
],
"all": [],
"none": [],
"failureSummary": null
},
{
"html": "<h1>Bar</h1>",
"target": [
"#ifr-foo",
"#foo-bar",
"h1"
],
"impact": null,
"any": [
{
"id": "has-visible-text",
"impact": "minor",
"message": "Element has text that is visible to screen readers",
"data": null,
"relatedNodes": []
}
],
"all": [],
"none": [],
"failureSummary": null
},
{
"html": "<h1>Baz</h1>",
"target": [
"#ifr-foo",
"#foo-bar",
"#bar-baz",
"h1"
],
"impact": null,
"any": [
{
"id": "has-visible-text",
"impact": "minor",
"message": "Element has text that is visible to screen readers",
"data": null,
"relatedNodes": []
}
],
"all": [],
"none": [],
"failureSummary": null
},
{
"html": "<h1>Baz</h1>",
"target": [
"#ifr-foo",
"#foo-baz",
"h1"
],
"impact": null,
"any": [
{
"id": "has-visible-text",
"impact": "minor",
"message": "Element has text that is visible to screen readers",
"data": null,
"relatedNodes": []
}
],
"all": [],
"none": [],
"failureSummary": null
},
{
"html": "<h1>Bar</h1>",
"target": [
"#ifr-bar",
"h1"
],
"impact": null,
"any": [
{
"id": "has-visible-text",
"impact": "minor",
"message": "Element has text that is visible to screen readers",
"data": null,
"relatedNodes": []
}
],
"all": [],
"none": [],
"failureSummary": null
},
{
"html": "<h1>Baz</h1>",
"target": [
"#ifr-bar",
"#bar-baz",
"h1"
],
"impact": null,
"any": [
{
"id": "has-visible-text",
"impact": "minor",
"message": "Element has text that is visible to screen readers",
"data": null,
"relatedNodes": []
}
],
"all": [],
"none": [],
"failureSummary": null
},
{
"html": "<h1>Baz</h1>",
"target": [
"#ifr-baz",
"h1"
],
"impact": null,
"any": [
{
"id": "has-visible-text",
"impact": "minor",
"message": "Element has text that is visible to screen readers",
"data": null,
"relatedNodes": []
}
],
"all": [],
"none": [],
"failureSummary": null
}
],
"url": null,
"createdDate": null
},
{
"id": "form-field-multiple-labels",
"description": "Ensure form field does not have multiple label elements",
"help": "Form field must not have multiple label elements",
"helpUrl": "https://dequeuniversity.com/rules/axe/4.10/form-field-multiple-labels?application=axeAPI",
"impact": "",
"tags": [
"cat.forms",
"wcag2a",
"wcag332",
"TTv5",
"TT5.c",
"EN-301-549",
"EN-9.3.3.2"
],
"nodes": [
{
"html": "<input type=\"text\">",
"target": [
"#ifr-foo",
"#foo-bar",
"#bar-baz",
"input"
],
"impact": null,
"any": [],
"all": [],
"none": [
{
"id": "multiple-label",
"impact": "moderate",
"message": "Form field does not have multiple label elements",
"data": null,
"relatedNodes": []
}
],
"failureSummary": null
},
{
"html": "<input type=\"text\">",
"target": [
"#ifr-foo",
"#foo-baz",
"input"
],
"impact": null,
"any": [],
"all": [],
"none": [
{
"id": "multiple-label",
"impact": "moderate",
"message": "Form field does not have multiple label elements",
"data": null,
"relatedNodes": []
}
],
"failureSummary": null
},
{
"html": "<input type=\"text\">",
"target": [
"#ifr-bar",
"#bar-baz",
"input"
],
"impact": null,
"any": [],
"all": [],
"none": [
{
"id": "multiple-label",
"impact": "moderate",
"message": "Form field does not have multiple label elements",
"data": null,
"relatedNodes": []
}
],
"failureSummary": null
},
{
"html": "<input type=\"text\">",
"target": [
"#ifr-baz",
"input"
],
"impact": null,
"any": [],
"all": [],
"none": [
{
"id": "multiple-label",
"impact": "moderate",
"message": "Form field does not have multiple label elements",
"data": null,
"relatedNodes": []
}
],
"failureSummary": null
}
],
"url": null,
"createdDate": null
},
{
"id": "frame-tested",
"description": "Ensure <iframe> and <frame> elements contain the axe-core script",
"help": "Frames should be tested with axe-core",
"helpUrl": "https://dequeuniversity.com/rules/axe/4.10/frame-tested?application=axeAPI",
"impact": "",
"tags": [
"cat.structure",
"best-practice",
"review-item"
],
"nodes": [
{
"html": "<iframe src=\"iframes/foo.html\" id=\"ifr-foo\" cd_frame_id_=\"8d0a624e866ec0bce080b0de86c16411\"></iframe>",
"target": [
"#ifr-foo"
],
"impact": null,
"any": [],
"all": [
{
"id": "frame-tested",
"impact": "critical",
"message": "The iframe was tested with axe-core",
"data": null,
"relatedNodes": []
}
],
"none": [],
"failureSummary": null
},
{
"html": "<iframe src=\"bar.html\" id=\"foo-bar\" cd_frame_id_=\"5830db61d325a1a554e5e72eb2ba2959\"></iframe>",
"target": [
"#ifr-foo",
"#foo-bar"
],
"impact": null,
"any": [],
"all": [
{
"id": "frame-tested",
"impact": "critical",
"message": "The iframe was tested with axe-core",
"data": null,
"relatedNodes": []
}
],
"none": [],
"failureSummary": null
},
{
"html": "<iframe src=\"baz.html\" id=\"bar-baz\" cd_frame_id_=\"59debf23e7a5b6ad200693d31b2c9a49\"></iframe>",
"target": [
"#ifr-foo",
"#foo-bar",
"#bar-baz"
],
"impact": null,
"any": [],
"all": [
{
"id": "frame-tested",
"impact": "critical",
"message": "The iframe was tested with axe-core",
"data": null,
"relatedNodes": []
}
],
"none": [],
"failureSummary": null
},
{
"html": "<iframe src=\"baz.html\" id=\"foo-baz\" cd_frame_id_=\"af6e22095931226bb0861676921f74ae\"></iframe>",
"target": [
"#ifr-foo",
"#foo-baz"
],
"impact": null,
"any": [],
"all": [
{
"id": "frame-tested",
"impact": "critical",
"message": "The iframe was tested with axe-core",
"data": null,
"relatedNodes": []
}
],
"none": [],
"failureSummary": null
},
{
"html": "<iframe src=\"iframes/bar.html\" id=\"ifr-bar\" cd_frame_id_=\"0050a23593b6247a27f14be37e6c3709\"></iframe>",
"target": [
"#ifr-bar"
],
"impact": null,
"any": [],
"all": [
{
"id": "frame-tested",
"impact": "critical",
"message": "The iframe was tested with axe-core",
"data": null,
"relatedNodes": []
}
],
"none": [],
"failureSummary": null
},
{
"html": "<iframe src=\"baz.html\" id=\"bar-baz\" cd_frame_id_=\"342f960ce54b8dc4c449eaaeb93dc693\"></iframe>",
"target": [
"#ifr-bar",
"#bar-baz"
],
"impact": null,
"any": [],
"all": [
{
"id": "frame-tested",
"impact": "critical",
"message": "The iframe was tested with axe-core",
"data": null,
"relatedNodes": []
}
],
"none": [],
"failureSummary": null
},
{
"html": "<iframe src=\"iframes/baz.html\" id=\"ifr-baz\" cd_frame_id_=\"a0710c56b8714f1663b978c3472af276\"></iframe>",
"target": [
"#ifr-baz"
],
"impact": null,
"any": [],
"all": [
{
"id": "frame-tested",
"impact": "critical",
"message": "The iframe was tested with axe-core",
"data": null,
"relatedNodes": []
}
],
"none": [],
"failureSummary": null
}
],
"url": null,
"createdDate": null
},
{
"id": "heading-order",
"description": "Ensure the order of headings is semantically correct",
"help": "Heading levels should only increase by one",
"helpUrl": "https://dequeuniversity.com/rules/axe/4.10/heading-order?application=axeAPI",
"impact": "",
"tags": [
"cat.semantics",
"best-practice"
],
"nodes": [
{
"html": "<h1>This page has nested frames!</h1>",
"target": [
"h1"
],
"impact": null,
"any": [
{
"id": "heading-order",
"impact": "moderate",
"message": "Heading order valid",
"data": {
"headingOrder": [
{
"ancestry": [
"html > body > h1:nth-child(1)"
],
"level": 1
},
{
"ancestry": [
"html > body > iframe:nth-child(2)"
],
"level": -1
},
{
"ancestry": [
"html > body > iframe:nth-child(3)"
],
"level": -1
},
{
"ancestry": [
"html > body > iframe:nth-child(4)"
],
"level": -1
}
]
},
"relatedNodes": []
}
],
"all": [],
"none": [],
"failureSummary": null
},
{
"html": "<h1>Foo</h1>",
"target": [
"#ifr-foo",
"h1"
],
"impact": null,
"any": [
{
"id": "heading-order",
"impact": "moderate",
"message": "Heading order valid",
"data": {
"headingOrder": [
{
"ancestry": [
"html > body > h1:nth-child(1)"
],
"level": 1
},
{
"ancestry": [
"html > body > iframe:nth-child(2)"
],
"level": -1
},
{
"ancestry": [
"html > body > iframe:nth-child(3)"
],
"level": -1
}
]
},
"relatedNodes": []
}
],
"all": [],
"none": [],
"failureSummary": null
},
{
"html": "<h1>Bar</h1>",
"target": [
"#ifr-foo",
"#foo-bar",
"h1"
],
"impact": null,
"any": [
{
"id": "heading-order",
"impact": "moderate",
"message": "Heading order valid",
"data": {
"headingOrder": [
{
"ancestry": [
"html > body > h1:nth-child(1)"
],
"level": 1
},
{
"ancestry": [
"html > body > iframe:nth-child(2)"
],
"level": -1
}
]
},
"relatedNodes": []
}
],
"all": [],
"none": [],
"failureSummary": null
},
{
"html": "<h1>Baz</h1>",
"target": [
"#ifr-foo",
"#foo-bar",
"#bar-baz",
"h1"
],
"impact": null,
"any": [
{
"id": "heading-order",
"impact": "moderate",
"message": "Heading order valid",
"data": {
"headingOrder": [
{
"ancestry": [
"html > body > h1:nth-child(1)"
],
"level": 1
}
]
},
"relatedNodes": []
}
],
"all": [],
"none": [],
"failureSummary": null
},
{
"html": "<h1>Baz</h1>",
"target": [
"#ifr-foo",
"#foo-baz",
"h1"
],
"impact": null,
"any": [
{
"id": "heading-order",
"impact": "moderate",
"message": "Heading order valid",
"data": {
"headingOrder": [
{
"ancestry": [
"html > body > h1:nth-child(1)"
],
"level": 1
}
]
},
"relatedNodes": []
}
],
"all": [],
"none": [],
"failureSummary": null
},
{
"html": "<h1>Bar</h1>",
"target": [
"#ifr-bar",
"h1"
],
"impact": null,
"any": [
{
"id": "heading-order",
"impact": "moderate",
"message": "Heading order valid",
"data": {
"headingOrder": [
{
"ancestry": [
"html > body > h1:nth-child(1)"
],
"level": 1
},
{
"ancestry": [
"html > body > iframe:nth-child(2)"
],
"level": -1
}
]
},
"relatedNodes": []
}
],
"all": [],
"none": [],
"failureSummary": null
},
{
"html": "<h1>Baz</h1>",
"target": [
"#ifr-bar",
"#bar-baz",
"h1"
],
"impact": null,
"any": [
{
"id": "heading-order",
"impact": "moderate",
"message": "Heading order valid",
"data": {
"headingOrder": [
{
"ancestry": [
"html > body > h1:nth-child(1)"
],
"level": 1
}
]
},
"relatedNodes": []
}
],
"all": [],
"none": [],
"failureSummary": null
},
{
"html": "<h1>Baz</h1>",
"target": [
"#ifr-baz",
"h1"
],
"impact": null,
"any": [
{
"id": "heading-order",
"impact": "moderate",
"message": "Heading order valid",
"data": {
"headingOrder": [
{
"ancestry": [
"html > body > h1:nth-child(1)"
],
"level": 1
}
]
},
"relatedNodes": []
}
],
"all": [],
"none": [],
"failureSummary": null
}
],
"url": null,
"createdDate": null
},
{
"id": "html-has-lang",
"description": "Ensure every HTML document has a lang attribute",
"help": "<html> element must have a lang attribute",
"helpUrl": "https://dequeuniversity.com/rules/axe/4.10/html-has-lang?application=axeAPI",
"impact": "",
"tags": [
"cat.language",
"wcag2a",
"wcag311",
"TTv5",
"TT11.a",
"EN-301-549",
"EN-9.3.1.1",
"ACT"
],
"nodes": [
{
"html": "<html lang=\"en\">",
"target": [
"html"
],
"impact": null,
"any": [
{
"id": "has-lang",
"impact": "serious",
"message": "The <html> element has a lang attribute",
"data": null,
"relatedNodes": []
}
],
"all": [],
"none": [],
"failureSummary": null
}
],
"url": null,
"createdDate": null
},
{
"id": "html-lang-valid",
"description": "Ensure the lang attribute of the <html> element has a valid value",
"help": "<html> element must have a valid value for the lang attribute",
"helpUrl": "https://dequeuniversity.com/rules/axe/4.10/html-lang-valid?application=axeAPI",
"impact": "",
"tags": [
"cat.language",
"wcag2a",
"wcag311",
"TTv5",
"TT11.a",
"EN-301-549",
"EN-9.3.1.1",
"ACT"
],
"nodes": [
{
"html": "<html lang=\"en\">",
"target": [
"html"
],
"impact": null,
"any": [],
"all": [],
"none": [
{
"id": "valid-lang",
"impact": "serious",
"message": "Value of lang attribute is included in the list of valid languages",
"data": null,
"relatedNodes": []
}
],
"failureSummary": null
},
{
"html": "<html lang=\"en\"><head>\n <title>Foo\n \n \n
Foo
\n \n \n \n",
"target": [
"#ifr-foo",
"html"
],
"impact": null,
"any": [],
"all": [],
"none": [
{
"id": "valid-lang",
"impact": "serious",
"message": "Value of lang attribute is included in the list of valid languages",
"data": null,
"relatedNodes": []
}
],
"failureSummary": null
},
{
"html": "\n Bar\n \n \n
Bar
\n \n \n",
"target": [
"#ifr-foo",
"#foo-bar",
"html"
],
"impact": null,
"any": [],
"all": [],
"none": [
{
"id": "valid-lang",
"impact": "serious",
"message": "Value of lang attribute is included in the list of valid languages",
"data": null,
"relatedNodes": []
}
],
"failureSummary": null
},
{
"html": "\n Baz\n \n \n
Baz
\n \n \n",
"target": [
"#ifr-foo",
"#foo-bar",
"#bar-baz",
"html"
],
"impact": null,
"any": [],
"all": [],
"none": [
{
"id": "valid-lang",
"impact": "serious",
"message": "Value of lang attribute is included in the list of valid languages",
"data": null,
"relatedNodes": []
}
],
"failureSummary": null
},
{
"html": "\n Baz\n \n \n
Baz
\n \n \n",
"target": [
"#ifr-foo",
"#foo-baz",
"html"
],
"impact": null,
"any": [],
"all": [],
"none": [
{
"id": "valid-lang",
"impact": "serious",
"message": "Value of lang attribute is included in the list of valid languages",
"data": null,
"relatedNodes": []
}
],
"failureSummary": null
},
{
"html": "\n Bar\n \n \n
Bar
\n \n \n",
"target": [
"#ifr-bar",
"html"
],
"impact": null,
"any": [],
"all": [],
"none": [
{
"id": "valid-lang",
"impact": "serious",
"message": "Value of lang attribute is included in the list of valid languages",
"data": null,
"relatedNodes": []
}
],
"failureSummary": null
},
{
"html": "\n Baz\n \n \n
Baz
\n \n \n",
"target": [
"#ifr-bar",
"#bar-baz",
"html"
],
"impact": null,
"any": [],
"all": [],
"none": [
{
"id": "valid-lang",
"impact": "serious",
"message": "Value of lang attribute is included in the list of valid languages",
"data": null,
"relatedNodes": []
}
],
"failureSummary": null
},
{
"html": "\n Baz\n \n \n
Baz
\n \n \n",
"target": [
"#ifr-baz",
"html"
],
"impact": null,
"any": [],
"all": [],
"none": [
{
"id": "valid-lang",
"impact": "serious",
"message": "Value of lang attribute is included in the list of valid languages",
"data": null,
"relatedNodes": []
}
],
"failureSummary": null
}
],
"url": null,
"createdDate": null
},
{
"id": "label-title-only",
"description": "Ensure that every form element has a visible label and is not solely labeled using hidden labels, or the title or aria-describedby attributes",
"help": "Form elements should have a visible label",
"helpUrl": "https://dequeuniversity.com/rules/axe/4.10/label-title-only?application=axeAPI",
"impact": "",
"tags": [
"cat.forms",
"best-practice"
],
"nodes": [
{
"html": "",
"target": [
"#ifr-foo",
"#foo-bar",
"#bar-baz",
"input"
],
"impact": null,
"any": [],
"all": [],
"none": [
{
"id": "title-only",
"impact": "serious",
"message": "Form element does not solely use title attribute for its label",
"data": null,
"relatedNodes": []
}
],
"failureSummary": null
},
{
"html": "",
"target": [
"#ifr-foo",
"#foo-baz",
"input"
],
"impact": null,
"any": [],
"all": [],
"none": [
{
"id": "title-only",
"impact": "serious",
"message": "Form element does not solely use title attribute for its label",
"data": null,
"relatedNodes": []
}
],
"failureSummary": null
},
{
"html": "",
"target": [
"#ifr-bar",
"#bar-baz",
"input"
],
"impact": null,
"any": [],
"all": [],
"none": [
{
"id": "title-only",
"impact": "serious",
"message": "Form element does not solely use title attribute for its label",
"data": null,
"relatedNodes": []
}
],
"failureSummary": null
},
{
"html": "",
"target": [
"#ifr-baz",
"input"
],
"impact": null,
"any": [],
"all": [],
"none": [
{
"id": "title-only",
"impact": "serious",
"message": "Form element does not solely use title attribute for its label",
"data": null,
"relatedNodes": []
}
],
"failureSummary": null
}
],
"url": null,
"createdDate": null
},
{
"id": "page-has-heading-one",
"description": "Ensure that the page, or at least one of its frames contains a level-one heading",
"help": "Page should contain a level-one heading",
"helpUrl": "https://dequeuniversity.com/rules/axe/4.10/page-has-heading-one?application=axeAPI",
"impact": "",
"tags": [
"cat.semantics",
"best-practice"
],
"nodes": [
{
"html": "",
"target": [
"html"
],
"impact": null,
"any": [],
"all": [
{
"id": "page-has-heading-one",
"impact": "moderate",
"message": "Page has at least one level-one heading",
"data": null,
"relatedNodes": [
{
"html": "
\n \n \n",
"target": [
"#ifr-baz",
"html"
],
"impact": null,
"any": [],
"all": [
{
"id": "page-has-heading-one",
"impact": "moderate",
"message": "Page has at least one level-one heading",
"data": null,
"relatedNodes": [
{
"html": "
Baz
",
"target": [
"#ifr-baz",
"h1"
]
}
]
}
],
"none": [],
"failureSummary": null
}
],
"url": null,
"createdDate": null
},
{
"id": "region",
"description": "Ensure all page content is contained by landmarks",
"help": "All page content should be contained by landmarks",
"helpUrl": "https://dequeuniversity.com/rules/axe/4.10/region?application=axeAPI",
"impact": "moderate",
"tags": [
"cat.keyboard",
"best-practice"
],
"nodes": [
{
"html": "",
"target": [
"#ifr-foo"
],
"impact": null,
"any": [
{
"id": "region",
"impact": "moderate",
"message": "All page content is contained by landmarks",
"data": {
"isIframe": true
},
"relatedNodes": []
}
],
"all": [],
"none": [],
"failureSummary": null
},
{
"html": "",
"target": [
"#ifr-foo",
"#foo-bar"
],
"impact": null,
"any": [
{
"id": "region",
"impact": "moderate",
"message": "All page content is contained by landmarks",
"data": {
"isIframe": true
},
"relatedNodes": []
}
],
"all": [],
"none": [],
"failureSummary": null
},
{
"html": "",
"target": [
"#ifr-foo",
"#foo-bar",
"#bar-baz"
],
"impact": null,
"any": [
{
"id": "region",
"impact": "moderate",
"message": "All page content is contained by landmarks",
"data": {
"isIframe": true
},
"relatedNodes": []
}
],
"all": [],
"none": [],
"failureSummary": null
},
{
"html": "",
"target": [
"#ifr-foo",
"#foo-baz"
],
"impact": null,
"any": [
{
"id": "region",
"impact": "moderate",
"message": "All page content is contained by landmarks",
"data": {
"isIframe": true
},
"relatedNodes": []
}
],
"all": [],
"none": [],
"failureSummary": null
},
{
"html": "",
"target": [
"#ifr-bar"
],
"impact": null,
"any": [
{
"id": "region",
"impact": "moderate",
"message": "All page content is contained by landmarks",
"data": {
"isIframe": true
},
"relatedNodes": []
}
],
"all": [],
"none": [],
"failureSummary": null
},
{
"html": "",
"target": [
"#ifr-bar",
"#bar-baz"
],
"impact": null,
"any": [
{
"id": "region",
"impact": "moderate",
"message": "All page content is contained by landmarks",
"data": {
"isIframe": true
},
"relatedNodes": []
}
],
"all": [],
"none": [],
"failureSummary": null
},
{
"html": "",
"target": [
"#ifr-baz"
],
"impact": null,
"any": [
{
"id": "region",
"impact": "moderate",
"message": "All page content is contained by landmarks",
"data": {
"isIframe": true
},
"relatedNodes": []
}
],
"all": [],
"none": [],
"failureSummary": null
}
],
"url": null,
"createdDate": null
}
],
"violations": [
{
"id": "frame-title",
"description": "Ensure ",
"target": [
"#ifr-foo"
],
"impact": "serious",
"any": [
{
"id": "non-empty-title",
"impact": "serious",
"message": "Element has no title attribute",
"data": {
"messageKey": "noAttr"
},
"relatedNodes": []
},
{
"id": "aria-label",
"impact": "serious",
"message": "aria-label attribute does not exist or is empty",
"data": null,
"relatedNodes": []
},
{
"id": "aria-labelledby",
"impact": "serious",
"message": "aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty",
"data": null,
"relatedNodes": []
},
{
"id": "presentational-role",
"impact": "serious",
"message": "Element's default semantics were not overridden with role=\"none\" or role=\"presentation\"",
"data": null,
"relatedNodes": []
}
],
"all": [],
"none": [],
"failureSummary": "Fix any of the following:\n Element has no title attribute\n aria-label attribute does not exist or is empty\n aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty\n Element's default semantics were not overridden with role=\"none\" or role=\"presentation\""
},
{
"html": "",
"target": [
"#ifr-foo",
"#foo-bar"
],
"impact": "serious",
"any": [
{
"id": "non-empty-title",
"impact": "serious",
"message": "Element has no title attribute",
"data": {
"messageKey": "noAttr"
},
"relatedNodes": []
},
{
"id": "aria-label",
"impact": "serious",
"message": "aria-label attribute does not exist or is empty",
"data": null,
"relatedNodes": []
},
{
"id": "aria-labelledby",
"impact": "serious",
"message": "aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty",
"data": null,
"relatedNodes": []
},
{
"id": "presentational-role",
"impact": "serious",
"message": "Element's default semantics were not overridden with role=\"none\" or role=\"presentation\"",
"data": null,
"relatedNodes": []
}
],
"all": [],
"none": [],
"failureSummary": "Fix any of the following:\n Element has no title attribute\n aria-label attribute does not exist or is empty\n aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty\n Element's default semantics were not overridden with role=\"none\" or role=\"presentation\""
},
{
"html": "",
"target": [
"#ifr-foo",
"#foo-bar",
"#bar-baz"
],
"impact": "serious",
"any": [
{
"id": "non-empty-title",
"impact": "serious",
"message": "Element has no title attribute",
"data": {
"messageKey": "noAttr"
},
"relatedNodes": []
},
{
"id": "aria-label",
"impact": "serious",
"message": "aria-label attribute does not exist or is empty",
"data": null,
"relatedNodes": []
},
{
"id": "aria-labelledby",
"impact": "serious",
"message": "aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty",
"data": null,
"relatedNodes": []
},
{
"id": "presentational-role",
"impact": "serious",
"message": "Element's default semantics were not overridden with role=\"none\" or role=\"presentation\"",
"data": null,
"relatedNodes": []
}
],
"all": [],
"none": [],
"failureSummary": "Fix any of the following:\n Element has no title attribute\n aria-label attribute does not exist or is empty\n aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty\n Element's default semantics were not overridden with role=\"none\" or role=\"presentation\""
},
{
"html": "",
"target": [
"#ifr-foo",
"#foo-baz"
],
"impact": "serious",
"any": [
{
"id": "non-empty-title",
"impact": "serious",
"message": "Element has no title attribute",
"data": {
"messageKey": "noAttr"
},
"relatedNodes": []
},
{
"id": "aria-label",
"impact": "serious",
"message": "aria-label attribute does not exist or is empty",
"data": null,
"relatedNodes": []
},
{
"id": "aria-labelledby",
"impact": "serious",
"message": "aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty",
"data": null,
"relatedNodes": []
},
{
"id": "presentational-role",
"impact": "serious",
"message": "Element's default semantics were not overridden with role=\"none\" or role=\"presentation\"",
"data": null,
"relatedNodes": []
}
],
"all": [],
"none": [],
"failureSummary": "Fix any of the following:\n Element has no title attribute\n aria-label attribute does not exist or is empty\n aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty\n Element's default semantics were not overridden with role=\"none\" or role=\"presentation\""
},
{
"html": "",
"target": [
"#ifr-bar"
],
"impact": "serious",
"any": [
{
"id": "non-empty-title",
"impact": "serious",
"message": "Element has no title attribute",
"data": {
"messageKey": "noAttr"
},
"relatedNodes": []
},
{
"id": "aria-label",
"impact": "serious",
"message": "aria-label attribute does not exist or is empty",
"data": null,
"relatedNodes": []
},
{
"id": "aria-labelledby",
"impact": "serious",
"message": "aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty",
"data": null,
"relatedNodes": []
},
{
"id": "presentational-role",
"impact": "serious",
"message": "Element's default semantics were not overridden with role=\"none\" or role=\"presentation\"",
"data": null,
"relatedNodes": []
}
],
"all": [],
"none": [],
"failureSummary": "Fix any of the following:\n Element has no title attribute\n aria-label attribute does not exist or is empty\n aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty\n Element's default semantics were not overridden with role=\"none\" or role=\"presentation\""
},
{
"html": "",
"target": [
"#ifr-bar",
"#bar-baz"
],
"impact": "serious",
"any": [
{
"id": "non-empty-title",
"impact": "serious",
"message": "Element has no title attribute",
"data": {
"messageKey": "noAttr"
},
"relatedNodes": []
},
{
"id": "aria-label",
"impact": "serious",
"message": "aria-label attribute does not exist or is empty",
"data": null,
"relatedNodes": []
},
{
"id": "aria-labelledby",
"impact": "serious",
"message": "aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty",
"data": null,
"relatedNodes": []
},
{
"id": "presentational-role",
"impact": "serious",
"message": "Element's default semantics were not overridden with role=\"none\" or role=\"presentation\"",
"data": null,
"relatedNodes": []
}
],
"all": [],
"none": [],
"failureSummary": "Fix any of the following:\n Element has no title attribute\n aria-label attribute does not exist or is empty\n aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty\n Element's default semantics were not overridden with role=\"none\" or role=\"presentation\""
},
{
"html": "",
"target": [
"#ifr-baz"
],
"impact": "serious",
"any": [
{
"id": "non-empty-title",
"impact": "serious",
"message": "Element has no title attribute",
"data": {
"messageKey": "noAttr"
},
"relatedNodes": []
},
{
"id": "aria-label",
"impact": "serious",
"message": "aria-label attribute does not exist or is empty",
"data": null,
"relatedNodes": []
},
{
"id": "aria-labelledby",
"impact": "serious",
"message": "aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty",
"data": null,
"relatedNodes": []
},
{
"id": "presentational-role",
"impact": "serious",
"message": "Element's default semantics were not overridden with role=\"none\" or role=\"presentation\"",
"data": null,
"relatedNodes": []
}
],
"all": [],
"none": [],
"failureSummary": "Fix any of the following:\n Element has no title attribute\n aria-label attribute does not exist or is empty\n aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty\n Element's default semantics were not overridden with role=\"none\" or role=\"presentation\""
}
],
"url": null,
"createdDate": null
},
{
"id": "label",
"description": "Ensure every form element has a label",
"help": "Form elements must have labels",
"helpUrl": "https://dequeuniversity.com/rules/axe/4.10/label?application=axeAPI",
"impact": "critical",
"tags": [
"cat.forms",
"wcag2a",
"wcag412",
"section508",
"section508.22.n",
"TTv5",
"TT5.c",
"EN-301-549",
"EN-9.4.1.2",
"ACT"
],
"nodes": [
{
"html": "",
"target": [
"#ifr-foo",
"#foo-bar",
"#bar-baz",
"input"
],
"impact": "critical",
"any": [
{
"id": "implicit-label",
"impact": "critical",
"message": "Form element does not have an implicit (wrapped)
Zidious
commented
2 hours ago
runLegacy()andrunPartial/finishRundo not produce the same results causing our test to fail as we assert that the axe-results generated are equal. Test in question: https://github.com/dequelabs/axe-core-maven-html/blob/3d13cc7662e7428c7217ba687e696a0622c9ac62/selenium/src/test/java/com/deque/html/axecore/selenium/Axe43xIntegrationTest.java#L257-L276There are few things happening here:
When legacy mode is enabled, selenium handles collecting all of the iframes for axe to inject: https://github.com/dequelabs/axe-core-maven-html/blob/3d13cc7662e7428c7217ba687e696a0622c9ac62/selenium/src/main/java/com/deque/html/axecore/extensions/WebDriverInjectorExtensions.java#L135-L171
Selenium now adds an attribute to each iframe called
cd_frame_idwhich is some unique UUID. This is the first difference, as forrunPartialwe do not rely on Selenium to collect all of the frames thus nocd_frame_idattribute being added onto each iframe. This causes the equality check between results to fail. An HTML snippet captured by axe:Moreover,
aria-hidden-bodyrule appears to only capture thebodyelement during a legacy run but everything including thebodyduring a normal run:Legacy
```json { "id": "aria-hidden-body", "description": "Ensure aria-hidden=\"true\" is not present on the document body.", "help": "aria-hidden=\"true\" must not be present on the document body", "helpUrl": "https://dequeuniversity.com/rules/axe/4.10/aria-hidden-body?application=axeAPI", "impact": "", "tags": [ "cat.aria", "wcag2a", "wcag131", "wcag412", "EN-301-549", "EN-9.1.3.1", "EN-9.4.1.2" ], "nodes": [ { "html": "", "target": [ "body" ], "impact": null, "any": [ { "id": "aria-hidden-body", "impact": "critical", "message": "No aria-hidden attribute is present on document body", "data": null, "relatedNodes": [] } ], "all": [], "none": [], "failureSummary": null } ], "url": null, "createdDate": null } ```Normal
```json { "id": "aria-hidden-body", "description": "Ensure aria-hidden=\"true\" is not present on the document body.", "help": "aria-hidden=\"true\" must not be present on the document body", "helpUrl": "https://dequeuniversity.com/rules/axe/4.10/aria-hidden-body?application=axeAPI", "impact": "", "tags": [ "cat.aria", "wcag2a", "wcag131", "wcag412", "EN-301-549", "EN-9.1.3.1", "EN-9.4.1.2" ], "nodes": [ { "html": "\nThis page has nested frames!
\n \n \n \n \n", "target": [ "body" ], "impact": null, "any": [ { "id": "aria-hidden-body", "impact": "critical", "message": "No aria-hidden attribute is present on document body", "data": null, "relatedNodes": [] } ], "all": [], "none": [], "failureSummary": null } ], "url": null, "createdDate": null } ```This does not happen with our @axe-core/webdriverjs test. The difference between the Java and TypeScript test is the Java test navigates to the page again before running a normal run: https://github.com/dequelabs/axe-core-maven-html/blob/3d13cc7662e7428c7217ba687e696a0622c9ac62/selenium/src/test/java/com/deque/html/axecore/selenium/Axe43xIntegrationTest.java#L267
Doing only one navigation in the WebDriverJS test causes some of the remanence of the legacy run notably the
cd_frame_idattribute and causes thearia-hidden-ruleto only capture the body. Removing the additional navigation in the Java test causes the test to pass. On the other hand, adding the additional navigation in the Typescript test causes the test to fail.Follow up that needs to happen:
bodyelement for legacy and everything insidebodyfor normal run. Second, how we can do true equality checks when selenium addscd_frame_idonto each iframe this fundamentally breaks our test(see below comments for full axe-results generated from each run)