search

web-platform-tests / interop-accessibility

Accessibility Testing for WPT
https://wpt.fyi/results/?q=label%3Aaccessibility
38 stars 5 forks source link

wpt.fyi results conflict in svg-aam/name/comp_host_language_label.html #142

Closed cookiecrook closed 1 week ago

cookiecrook commented 3 weeks ago

WPT.fyi lists a Safari failure on subtest "[xlink:title][href] > rect" but the Safari history shows it passing always, which aligns with our local testing... The conflicting results listed on the same wpt.fyi page can't both be true.

https://wpt.fyi/results/svg-aam/name/comp_host_language_label.html?label=master&label=experimental&aligned&view=interop&q=label%3Ainterop-2024-accessibility

Therefore I am assuming this is an error in how WPT.fyi tracks the results, rather than in the actual result.

If https://github.com/web-platform-tests/wpt.fyi/issues/3957 can't be resolved and we continue to receive flaky results, it may be good to move this test or set of tests to a .tentative file.

cookiecrook commented 3 weeks ago

FWIW, this is no longer showing an error immediately after filing. 🤷

cookiecrook commented 3 weeks ago

Cloned issue to wpt.fyi.

cookiecrook commented 3 weeks ago

Keeping this open to discuss at the monthly, but this is intermittently occurring on multiple subtests and multiple files. Removing the offending test no longer seems to be a valid option.

https://wpt.fyi/results/accname/name/comp_labelledby.html?label=master&label=experimental&aligned&view=interop&q=label%3Ainterop-2024-accessibility

I will close this after raising it to the Interop Accessibility in Sept.

cookiecrook commented 1 week ago

Discussed with the Interop group. Closing since there's nothing more to be done in this project.