Headings within legends are not exposed correctly in JAWS

NickColley commented 4 years ago

Reported by an external audit.

There is an issue with heading elements inside legends in the following assistive technologies.

Heading elements inside legends are associated and read out with the relevant fieldset, but are not surfaced as headings themselves on the page.

For example: when navigating via headings (pressing the ‘h’ key on a page), you would expect the heading to be announced (for example “heading level 1, What is your nationality?”. We instead hear “there are no headings on this page”.

JAWS (observed in JAWS 18+)

Reads ‘there are no headings on this page’
Not reading in context as a heading only a legend
Using ins F6 it is reading ‘no headings found’
Results are the same with CSS turned off

NVDA

Note: the same issue was originally raised against NVDA too, saying that NVDA read out ‘no next heading’ when navigating via headings. However, in Hanna’s investigation in 2019 and my own today (10/11/2021) we were both unable to replicate.

ZoomText

ZoomText’s own audio feedback also does not read the heading when placed within the <legend>

kellylee-gds commented 4 years ago

@nickcolley should we add the audit label to this so we can track?

hannalaakso commented 4 years ago

Did some investigation: https://docs.google.com/spreadsheets/d/1Tt89bAoleB0trw8qYsfgD4I3s7oQN4eYBrG6ukP__W8/edit?usp=sharing (publicly viewable spreadsheet)

Main points:

There seems to be an issue in Jaws 2018 (and 2019, as reported by DAC), while this works in Jaws 17 and 18. Will do a bit more digging.
Cycling through headings in NVDA with "Shift + H" behaves inconsistently, I don't think headings within legends are the issue.
Being able to reproduce "Ins + F6" issue in Jaws 18/2018 pending due with problem on Empathy Lab laptop.
We might need to go back to DAC to ask:
1. How they produced the ZoomText bug.
2. What is meant by "reading in context" in Jaws.

hannalaakso commented 4 years ago

There seems to be an issue in Jaws 2018 (and 2019, as reported by DAC), while this works in Jaws 17 and 18. Will do a bit more digging.

There are two reported open bugs on Jaws 2018/2019 that appears to be causing the heading within legend issues: https://github.com/FreedomScientific/VFO-standards-support/issues/100 https://github.com/FreedomScientific/VFO-standards-support/issues/315

Cycling through headings in NVDA with "Shift + H" behaves inconsistently, I don't think headings within legends are the issue.

Headings within legends are announced correctly, no further action required (NVDA appears to cycle through the headings backwards so you need to be focused on an element after a heading for Shift + H. This behaviour wasn't specific to the Design System pages.)

Being able to reproduce "Ins + F6" issue in Jaws 18/2018 pending due with problem on Empathy Lab laptop.

Could now test this. Jaws 18 works correctly, Jaws 2018 doesn't announce headings within legends, most likely linked to above reported Jaws bugs.

hannalaakso commented 4 years ago

Outcome of investigation

The issues with Jaws 2018/2019 have been reported to the vendor:

I've added our use case to those cards in case it helps the JAWS team to prioritise them.

I experimented fixing the issue with aria-attributes (role="heading" and aria-level="1") but this didn't change the Jaws announcements.

We might have to consider changing the pattern markup in a major way so resolve this.

Given that this is a known vendor issue and a recent regression (Jaws 17 and 18 are not impacted), we're not going to take further action now and I'm going to close the issue. If we get further feedback on this we'll reconsider the approach.

Work split out

As part of https://github.com/alphagov/design-system-team-internal/issues/160 we're going to contact DAC to ask about:

How they produced the ZoomText bug
What is meant by "reading in context" in Jaws

If any more work comes out of that we'll raise a new issue.

NickColley commented 4 years ago

Some instructions on how to reproduce this from the auditors:

Select the ‘Reader’ toolbar tab which displays toolbar controls for ZoomText's screen reading features and turn on (Caps Lock + Alt + Enter)

then when you hover the mouse over elements they will read back to you.

NickColley commented 4 years ago

Re-opening this for us to have another go at trying to reproduce the ZoomText issues.

36degrees commented 4 years ago

Let's timebox this – if we can reproduce it within an hour, let's report it. Otherwise, we'll close this.

36degrees commented 3 years ago

I can reproduce this with ZoomText on http://govuk-frontend-review.herokuapp.com/components/checkboxes/with-legend-as-a-page-heading/preview, but only in IE11.

lfdebrux commented 3 years ago

This was recently raised as an issue in an accessibility audit for a couple of services; one in DfE and one in the NHS. See x-gov Slack conversation https://ukgovernmentdigital.slack.com/archives/C28AJTE07/p1631609907224400.

andymantell commented 3 years ago

Before that thread gets archived in the X-Gov Slack, I'll say that the NHS service in question was 111 online. We're going to be investigating it a bit further in the near future and will report back if we get anywhere. Will try and get hold of the JAWS team and see if we can make any headway.

It seems to me though that with little movement since 2018 on that issue that it might be time to try a different markup pattern? It's not limited to IE11 as was reported in this ticket originally - the participant in our UR was running JAWS 2021 with Chrome and reported that no headings. So the impact on users is greater than it was when this ticket was raised.

(See also https://github.com/FreedomScientific/VFO-standards-support/issues/549 for confirmation that it's no longer just IE11)

andymantell commented 2 years ago

In the interests of not duplicating effort, I just thought I'd let you know I'm looking at a potential workaround, and as much testing as I can over here:

https://github.com/nhsuk/nhsuk-frontend/issues/778

If it tests well I'll let you know, and maybe someone from your team might be able to poke it as well, and see if you can forsee any issues. It basically involves adding role="heading" aria-level="1" onto the <legend> and removing the nested <h1>. It's testing well so far with JAWS, NVDA and TalkBack, but I've got more combinations to go...

Apparently the JAWS team is looking into it (See https://github.com/FreedomScientific/VFO-standards-support/issues/549#issuecomment-938570954) but we don't know a timeline for it. I've submitted a support request as well to see if they could give me some idea of timescales.

andymantell commented 2 years ago

Using the following markup pattern has tested well so far:

<legend class="nhsuk-fieldset__legend nhsuk-fieldset__legend--l" role="heading" aria-level="1">
  <span class="nhsuk-fieldset__heading">What is your address?</span>
</legend>

Tested in:

My test cases are at https://nhs-duec.github.io/headings-inside-legends-test-cases/ The above markup is obviously nhsuk-frontend focused but the underlying pattern is the same.

I wonder, would someone from the govuk-frontend side be able to do some testing around this pattern too? My testing has gone well so far but the solution is slightly "exotic" so I'm a little nervous about it. Currently it's invalid html but as per https://github.com/w3c/html-aria/issues/350 - Scott O'Hara is investigating making the pattern valid.

andymantell commented 2 years ago

@adamliptrot-oc was able to test MacOS / Voiceover for me as I can't test that platform - but unfortunately it appears that the heading was not accessible when using the role="heading". Which is probably enough to make this pattern unviable :-/

Fenwick17 commented 2 years ago

@andymantell Quick voiceover test shows that Safari can't detect the heading but Firefox with voiceover can detect it.

andymantell commented 2 years ago

Thanks @Fenwick17 - that's consistent with the conclusion here too:

https://github.com/w3c/html-aria/issues/350#issuecomment-940006805

i.e. that there's a webkit bug here. Frustrating - we're stuck between a JAWS bug and a Webkit bug 😞

vanitabarrett commented 2 years ago

Hi @andymantell

Thanks for raising this issue again and digging into it a bit!

Just to summarise what was discussed on Slack (before it gets lost), it sounds like this issue has been observed causing an issue for at least one participant in user research done on the NHS 111 service. In this case the participant was quite confused by the lack of heading, but was eventually able to complete the task by changing their behaviour.

On the role=“heading” approach - as a team I think we’re a bit wary of implementing that as a fix while roles are still invalid on the legend element, as it’d be flagged as invalid HTML to our users and it feels like a bit of a backwards step to replace valid HTML for a bug which really should be fixed by JAWS.

We did briefly discuss an alternative approach with visually hidden text, which was originally used in GOV.UK Elements before we changed to our current approach. You can read a bit more about why we switched away from that approach here https://github.com/alphagov/govuk_elements/pull/507

I think our preference for now, especially given the latest conversation in https://github.com/FreedomScientific/VFO-standards-support/issues/549#issuecomment-938570954, is to wait and see if anything comes out of the report that JAWS are looking into this issue now. If it continues to look like there’s no progress and/or if role on legends becomes valid, that might be a good time for us to revisit this. For any change, we’d be keen to test with users to make sure the change is beneficial for the majority of users. But we’re also keen to see what approach NHS takes and if there’s anything we can learn from you!

andymantell commented 2 years ago

Thanks @vanitabarrett

The approach we're planning to take on 111 online is to revert to the duplication of <h1> and <legend> by unnesting them - on the basis that we feel the mild annoyance of duplication is better than the confusion of a missing heading.

We then plan to undo this change as soon as the JAWS bug is resolved, or an alternative pattern such as the role="heading" approach becomes valid and works everywhere (which it currently doesn't - it's broken in MacOS Safari / Voiceover).

Completely agree with not rolling something like this out to govuk-frontend though. It's easy for us within our service to switch patterns on a whim, but you wouldn't want to be switching back and forwards at the design system level. Similarly, there is no planned change to nhsuk-frontend. This will be constrained to the service I am working on (111 online) for the moment.

lfdebrux commented 2 years ago

@andymantell yet another wrinkle, a user has reported that their service failed an accessibility audit because the heading was read out twice on a question page: https://github.com/alphagov/govuk-design-system-backlog/issues/58#issuecomment-945603738

andymantell commented 2 years ago

@lfdebrux read out twice with the current nested heading / legend approach? I still don't have access to iOS voiceover at the moment, but have we been able to reproduce that?

lfdebrux commented 2 years ago

@andymantell Yes. I can reproduce with Safari + VoiceOver on MacOS, it reads out something slightly different but it seems to be the same behaviour. A different person who reproduced it using Safari on iOS reported VoiceOver said

Where do you live? Heading level one. Where do you live? Landmark. Form start.

https://ukgovernmentdigital.slack.com/archives/C28AJTE07/p1634630243083200?thread_ts=1634593557.080000&cid=C28AJTE07

vanitabarrett commented 2 years ago

As this issue has been hanging around for a while and we’ve had reports from different people on the issues they’re seeing in different browser and assistive technology combinations, we thought it would be useful to clarify the impact of this issue.

This issue applies to heading elements within legends. Following the GOV.UK Design System guidance, this should only apply to question pages following the one-question-per-page pattern.

Impact

From our testing, the text within headings inside legends are associated and read out with the relevant fieldset, but are not surfaced as headings themselves on the page.

A user will still hear the text within the heading when:

navigating sequentially through the page and landing on the fieldset
tabbing through to the first checkbox

A user will not be able to navigate to the heading using the h key or see it in the list of headings when using INSERT+F6. If it is the only heading on the page, they will hear “there are no headings on this page”.

Technologies affected

Having tested this with different browsers and assistive technologies, we think the issue only affects JAWS with IE11, Edge and Chrome.

I was unable to replicate the issue in NVDA 2021.2, ZoomText 2021 or Voiceover.

⏩ View full testing spreadsheet

Next steps

The HTML spec states that it is valid for a legend to contain a heading. We therefore consider our implementation to be valid and that this issue is a bug in JAWS, especially given this implementation is read out correctly by other assistive technologies.

This issue was raised with Freedom Scientific in 2018 and we fed into this issue in 2019 when it was raised in our own external audit. We have also recently had an indication that this issue is something Freedom Scientific are aware of and working on - we will continue to try and push this where we can.

Note: We are not regular users of assistive technologies and we’ve drawn these conclusions from our own testing as we haven’t observed this causing issues for users in user research sessions. If you’ve used this pattern, we’d be interested in hearing your research findings.

vanitabarrett commented 2 years ago

Update: We've had an update from Freedom Scientific which suggests that this bug will be looked at as part of their 2022 cycle (i.e: released as an update at some point to version 2022). Unfortunately the update was quite vague, so they weren't able to give us any timescales on this, but it at least gives us some idea of when to start chasing again if we don't hear/see anything in the near future.

andymantell commented 2 years ago

I have tested this with JAWS 2022.2202.38 - video below...

Some observations:

Good ✔️

The H key now correctly announces the heading
The heading now appears correctly in the heading list dialog (unfortunately not shown on that video - I think my screen capture tool only captured the Chrome window and not the dialog that appeared)
The heading is announced as you navigate naturally through the page

Less good 👎🏻

Content duplication - if you listen to the videos you will hear it say "Group start this is a heading and a legend. Heading level one this is a heading and a legend"

So broadly speaking the issue is "fixed". Whether or not the duplication that we now get is something we're happy with is another discussion perhaps.

https://user-images.githubusercontent.com/134642/156594887-92022d42-f1f4-4eda-9609-0c22c2b13379.mp4

adamliptrot-oc commented 2 years ago

I think you have to expect the duplication as it is performing two tasks as the accessible name of the group and as the heading. How it achieves communicating that is down to the way the screen-reader works.

NVDA reports the following when moving through the page (so less verbose than JAWS):

"Grouping heading level 1 [legend/h1 title]"

When jumping direct to the heading JAWS just reports the heading:

"[legend/h1 title] heading level 1"

but with NVDA when jumping direct to the heading it also includes the group information:

"[legend/h1 title], Grouping [legend/h1 title] heading level 1"

andymantell commented 2 years ago

I think you're right - this is just how it is. In which case I think this is something that can be closed off now....

vanitabarrett commented 2 years ago

@andymantell When were you seeing the duplication? I'm doing my own testing in JAWS, as ours is updated to the latest version now, but I can't seem to replicate that behaviour.

andymantell commented 2 years ago

@vanitabarrett What page are you looking at? I can do you a video of my JAWS reading the same page...

vanitabarrett commented 2 years ago

@andymantell I've got access to JAWS 2022 now, so I was testing with the page we used last time to reproduce the issue and also tried it with your previous test cases but didn't seem to have any luck... I tried navigating to the heading and also just letting JAWS read out the full page.

andymantell commented 2 years ago

@vanitabarrett I've recorded a video of it here. I pressed refresh and then didn't touch the keyboard or mouse, so this is just JAWS reading from top to bottom. It takes a while to get to the point but eventually it will say "Group start what is your nationality? Heading level one what is your nationality?"

Pardon the question, but are you definitely running 2022? When I installed it, I ended up with both 2021 and 2022 on my machine and I have accidentally run the 2021 version on more than one occasion.

https://user-images.githubusercontent.com/134642/159735732-161c0d82-9e7c-4aca-bdde-882b7d88bae4.mp4

vanitabarrett commented 2 years ago

Thanks for clarifying and posting the video @andymantell - I was running the right version of JAWS, I just misunderstood the issue the first time I read it, I think! Can confirm I'm seeing the same behaviour but like you said I think that might be expected. And it definitely seems better behaviour than what we had previously.

vanitabarrett commented 2 years ago

Freedom Scientific fixed this bug in the December update for JAWS 2022, version 2022.2112.24. I’ve tested this version with the example we tested previously to reproduce the issue and can confirm that it seems to be working as expected (headings are announced and picked up by pressing the ‘h’ key or INSERT+F6) in Chrome, Firefox and Edge.

⏩ View full testing spreadsheet

As noted by Freedom Scientific and in my testing, this issue has not been and will not be fixed in IE11. Given that we’ve seen elsewhere that there’s been a recent decline in JAWS+Internet Explorer usage and I don’t think there’s any easy changes we can make on our side to improve how this is read for IE11 users, I’m going to suggest filing that away as a known issue and closing this ticket.

vanitabarrett commented 2 years ago

I’ve had a chat with @davidc-gds about the double heading announcement, and we agree that it’s difficult to know whether it’s an issue for anyone. We confirmed it shows up as an issue in NVDA too, so it’s not specific to JAWS.

We’re pretty confident that the HTML we’ve got now is the best approach: it’s tested well, it’s valid HTML and generally seems the best performing option.

Given that this issue has already had quite a lot of community involvement and there doesn’t seem to be any obvious alternatives that work better for all screenreaders, I’m going to close this issue as fixed. If we hear reports of the double heading announcement causing issues, we can explore it as a new issue.

alphagov / govuk-frontend