search

w3c / aria

Accessible Rich Internet Applications (WAI-ARIA)
https://w3c.github.io/aria/
Other
654 stars 125 forks source link

WPT tests related to sectionheader and sectionfooter need to be updated #2295

Closed tranjocelyn closed 3 months ago

tranjocelyn commented 3 months ago

I have the implementation for the new sectionheader and sectionfooter roles ready for Chrome, but it's currently failing the following subtests in WPT:

el-footer, el-header, and el-header-ancestormain in wpt/html-aam/roles-contextual.html footer element scoped to main element is generic in wpt/wai-aria/role/contextual-roles.html

I believe these subtests should be expecting the sectionheader/sectionfooter roles instead of generic.

spectranaut commented 3 months ago

I believe you are right! So I think we should update them.

However, this will cause these tests to fail on browser that haven't implemented these roles... We have a tentative file for the roles, I think what needs to happen is that we need to update the tests and move them into this file: https://github.com/web-platform-tests/wpt/pull/4614/files

But I'm not sure if this is always how we should go forward and I'd like @cookiecrook to weigh in. I think it's common for tests to be added for new features to WPT, and for the test to simply fail until it's implemented. It seems like a lot of extra work to first make a tentative file, and then when all the browsers pass, no long mark it as tentative, and there is already quite a bit of work to get new features landed in ARIA. I'm not sure I totally understand the repercussions of an interop score changing over time, specifically, going up and and down. Won't this always be the case, as new features are developed? Don't we want to see these signals?

cookiecrook commented 3 months ago

Yes, normally changing the existing file is fine...

However in this case, both of those files are tracked as part of the Interop 2024 Accessibility Focus Area, which is scored, and the score is archived at the end of the year. We try to avoid introducing new subtests mid-year, since they impact the score outside the agreed-upon list of test files (including their subtests) that were finalized in January.

The extra editorial work of moving new tests to a .tentative file is only expected in scenarios where the addition would make an inadvertent change to the current Focus Area scores. After the end of Interop 2024, we'll have an editorial task to merge these back in to the non-tentative files.

I think the new ARIA editorial process is also in-line with leaving tests "tentative" until two implementations pass, though I am fine being a bit more flexible on that expectation.

cookiecrook commented 3 months ago

So in this case, the failing tests should be moved to tentative and updated to unblock implementation of the new sectionheader/sectionfooter roles.

cookiecrook commented 3 months ago

Refiled as https://github.com/web-platform-tests/interop-accessibility/issues/136. Closing.

cookiecrook commented 3 months ago

@tranjocelyn talk to @chrishtr about whether you should be a team member in the WPT repo so we can assign you and request your review on future tests. Thanks for the contributions!

chrishtr commented 3 months ago

I think @tranjocelyn should already be a team member.

cookiecrook commented 3 months ago

FYI I don't see autocomplete options for "tranj" in comments, reviewers, or assignees in https://github.com/web-platform-tests/interop-accessibility/issues/136

foolip commented 3 months ago

I've added @tranjocelyn to the interop team now, and it looks like it'll be possible to assign issues. (@tranjocelyn, you were already and org member and in the reviewers team, which I first thought would be enough, but it wasn't.)