dequelabs / axe-core-npm

Mozilla Public License 2.0
577 stars 65 forks source link

fix(playwright): skip unloaded iframes #1060

Closed straker closed 1 month ago

straker commented 1 month ago

Pulled out of https://github.com/dequelabs/axe-core-npm/pull/1048. Starting in Playwright 1.41.0 (using Chrome version 121.0.6167.57) Playwright is no longer loading the lazy loaded iframe that is out of view. Updated the version of Playwright and updated the tests to now account for this.

QA notes: Test a lazy loaded iframe with Playwright > 1.41.0 and ensure Playwright will complete an analyze of the page

padmavemulapati commented 1 month ago

Verified with the latest canary-build (4.7.4-62fd241.0) Lazy-loaded iframes are scanning properly now, not hanging,

1. Here is iframes snippets: lazyloadedIframe.html ``` Lazy Loading Iframe Root

iframe context test

```
2. for this iframes - directory has a lazy.html file too ./iframes.lazy.html ``` Lazy

Lazy

```
3. for Playwright 1.44.0 Version and v1.42.1 : TestScript: ``` const { AxeBuilder } = require('@axe-core/playwright') const playwright = require('playwright') const fs = require('fs') const axe = require('axe-core') ;(async () => { const browser = await playwright.chromium.launch({ headless: true }) const context = await browser.newContext() const page = await context.newPage() await page.goto('http://localhost:8081/iframesfixture.html') try { const results = await new AxeBuilder({page}).analyze(); fs.writeFileSync('./lazyloadedIframes144.json', JSON.stringify(results, null, 4)) } catch (e) { console.error(e) } await browser.close() })() ``` Result for 1.44.0: ![image](https://github.com/dequelabs/axe-core-npm/assets/47409579/22b519e9-c218-43c3-9e77-1b6416490af2) Result for 1.42.1: ![image](https://github.com/dequelabs/axe-core-npm/assets/47409579/9d64d502-4fcd-4f70-8a1d-29ee1d3de712)
4. For Puppeteer v22.9.0 and v22.6.3. test-script: ``` const { AxePuppeteer } = require('@axe-core/puppeteer'); const puppeteer = require('puppeteer'); const fs = require('fs'); (async () => { const browser = await puppeteer.launch({ headless: false }); const page = await browser.newPage(); await page.setBypassCSP(true); await page.goto('http://localhost:8081/iframesfixture.html'); const results = await new AxePuppeteer(page).analyze(); fs.writeFile("./lazyframes_frametested.json", JSON.stringify(results, null,4),(err)=> {if(err){ console.error(err); return; }; console.log("File has been created"); }) await page.close(); await browser.close(); })(); ``` Results: For 22.9.0 ![image](https://github.com/dequelabs/axe-core-npm/assets/47409579/13708859-c7b3-43e8-9d63-7189a105b161) For 22.6.3 ![image](https://github.com/dequelabs/axe-core-npm/assets/47409579/845a6da3-5f8f-4124-8b93-924759245d9c)
5.for webdriverjs - selenium-webdriver v4.21.0 & v4.19.0: Test-script: ``` const { AxeBuilder } = require('@axe-core/webdriverjs'); const { Builder } = require('selenium-webdriver'); const chrome = require('selenium-webdriver/chrome'); const fs = require('fs'); (async () => { const driver = new Builder() .forBrowser('chrome') .setChromeOptions(new chrome.Options().headless()) .build(); await driver.get('http://localhost:8081/iframesfixture.html'); try { const results = await new AxeBuilder(driver).analyze(); fs.writeFile("./lazyiframestest.json", JSON.stringify(results, null,4),(err)=> {if(err){ console.error(err); return; }; console.log("File has been created"); }) } catch (e) { // do something with the error } await driver.quit(); })(); ``` Result: For 4.21.0 ![image](https://github.com/dequelabs/axe-core-npm/assets/47409579/e4a7090a-9bd5-4817-8365-e6bfef91929e) For 4.19.0 ![image](https://github.com/dequelabs/axe-core-npm/assets/47409579/c59a8b48-9710-48f1-8b92-87b8ed44d09f)
5. for CLI (selenium-webdriver v4.21.0 & 4.19.0)- #### command is `axe http://localhost:8081/iframesfixture.html --save ./iframesresult1 ` for selenium webdriver v4.21.0 ![image](https://github.com/dequelabs/axe-core-npm/assets/47409579/34b10953-251f-432c-8885-7f221a135195) for selenenium webdriver v4.19.0 ![image](https://github.com/dequelabs/axe-core-npm/assets/47409579/a87b1883-0c96-432e-b26f-673e87e93e58)
6. for webdriverio: TestScript: ``` const AxeBuilder = require('@axe-core/webdriverio').default; const { remote } = require('webdriverio'); const fs = require('fs'); (async () => { const client = await remote({ logLevel: 'error', capabilities: { browserName: 'chrome' } }); await client.url('http://localhost:8081/iframesfixture.html'); const builder = new AxeBuilder({ client }); const results = await builder.analyze(); fs.writeFile("./lazyiframestest.json", JSON.stringify(results, null,4),(err)=> {if(err){ console.error(err); return; }; console.log(results); }) client.deleteSession(); })(); ``` Result: for 8.36.1 ![image](https://github.com/dequelabs/axe-core-npm/assets/47409579/d2b8c506-25c4-45b4-b0cb-83ccb1357d35) for 8.35.1 ![image](https://github.com/dequelabs/axe-core-npm/assets/47409579/13677dc1-aad6-460f-b194-15f25987eab7)

Environment:

Label Value
Product axe-core-npm
Version 4.9.2-a2f07c5.0
Playwright 1.44.0 & 1.42.1
Puppeteer 22.9.0 & 22.6.3
WebdriverIO 8.36.1 & 8.35.1
Selenium-webdriver(WebdriverJS & CLI) v4.21.0 & 4.19.0
OS-Details MAC- M1-14.4 & Windows11
BrowserDetails Chrome Version 124.0.1 (Official Build) (64-bit) & Firefox 125.0.3