Open mcking65 opened 1 year ago
@mcking65, should this be even 2 issues? One the issue of the tests failing b/c of Windows-based assumptions, but the other of the remaining 2 tests that fail on macOS for ... 🤷🏽♀️.
The ARIA Authoring Practices (APG) Task Force just discussed Running tests on macOS
.
@frozenzia Before I can move on with this task, I just want to align on your Mac environment for running the regression tests. In test/index.js
I had to add the following to be able to run the regression tests:
...
const firefox = require('selenium-webdriver/firefox');
...
let options = new firefox.Options();
options.addArguments('-headless');
options.setBinary('/Applications/Firefox.app/Contents/MacOS/firefox');
If you have any workarounds to this, please let me know. I also wanted to align that we are seeing the same regression results. My regression output without the proposed changes are as follows - not sure if they have changed:
9 tests failed
6 known failures
29 tests remained pending after a timeout
I can confirm the 6 known failures. As for the other results, I'd like to make sure that we match. From my understanding, there were 5 failing tests upon the last comment in June, but now, there are 9 failing tests.
@evmiguel Sorry for the delayed response - I'll try to get to this in the next day or two and get back to you.
@evmiguel, alright, sorry, finally got to this. Using main branch, which had as its last commit 81d807dfa6dc0eea7f455dfd540af0c363845274 (from Feb 26, 2024). I have not had to touch test/index.js
to get this working. I still get 5 tests failed, 6 known failures. However, with the following difference:
In June the failing cases were in:
Now the failing cases are in:
AND, again if I make the same change to toolbar_toolbar.js
, specifically at lines 1119 and 1211:
async (t) => {
let textarea = await t.context.session.findElement(By.css('textarea'));
await textarea.sendKeys(Key.chord(Key.CONTROL, 'a'));
+ await textarea.sendKeys(Key.chord(Key.COMMAND, 'a')); //macOS
let originalText = await textarea.getAttribute('value');
async (t) => {
let textarea = await t.context.session.findElement(By.css('textarea'));
await textarea.sendKeys(Key.chord(Key.CONTROL, 'a'));
+ await textarea.sendKeys(Key.chord(Key.COMMAND, 'a')); //macOS
let originalText = await textarea.getAttribute('value');
With that change there are only 2 tests failed, 6 known failures. The 2 failed tests are in:
@frozenzia Thanks for getting back to me. I like your idea of detecting OS for managing edit fields. Are there any other tests you found that require this other than toolbar_toolbar.js
?
@frozenzia wrote:
In
toolbar_toolbar.js
, there are a couple of tests where CTRL+A are currently used to select all the text in a textarea. This does nothing on a Mac, and copying similar code fromalertdialog_alertdialog.js
, I've added an extra line to do the selecting in case the test is run on a Mac. After making this one and only change, running the tests still results in 2 test failures (+ 6 known failures):If I DON'T add those 2 lines in the toolbar test, I actually end up with 5 tests failed:
Originally posted by @frozenzia in https://github.com/w3c/aria-practices/issues/2628#issuecomment-1601967708