Iota-School / notebooks-for-all

https://iota-school.github.io/notebooks-for-all/
Other
18 stars 6 forks source link

add the ability to tab through cells (+labelling) #34

Closed tonyfast closed 1 year ago

tonyfast commented 1 year ago

this branch adds probably too much, but i was on a roll. this changes add:

tonyfast commented 1 year ago

@smythp @isabela-pf @Jenneh i'd love it if y'all could test ride a recent version on our work. this pull request make some of the near term changes we talked about like cell labels and tabbing. the implemented is not formalized yet, but we can test the generated output. to me, an inexperienced screen reader user, the tabbing experience seems to be improved. i'd love to know how y'all feel.

https://iota-school.github.io/notebooks-for-all/user-tests/2-content/Imaging_Sky_Background_Estimation.html.html5.html

of course, there is room to revert these changes. i'm going to put everything behind flags eventually. we can mix and match the solutions we choose.

tonyfast commented 1 year ago

when adding the tabs we need to include a visual style too... i don't know a subtle way to do this. in fact, i don't know subtle.

smythp commented 1 year ago

OK, tried it out, and the tab indexing works, which is pretty great. It currently moves from cell to cell, also landing on elements that typically are registered for tab indexing, like links.

However, I do think we have some issues with the labeling. There are two main issues.

First, the labeling behavior is inconsistent among major screen readers. On JAWS, it seems to work as intended (seems like it speaks headings when headings are present, input and a number with code cells, and markdown when no headings are present, more on that below). It also behaves as expected using the (pretty terrible but worth testing with) Orca on Linux. Using NVDA, however, when tabbing through cells, no output is presented at all, which is pretty disorienting when tabbing through. That is, when tabbing through the notebook using NVDA, it moves to the next cell, but doesn't speak anything aloud.

I can't say definitively why this behavior is occurring, but one strong possibility is that the labels are on an article element. From this page:

The aria-label and aria-labelledby attributes can be used to give an element an accessible name. The aria-describedby attribute can be used to give an element an accessible description. Not all elements can be given an accessible name and/or description though.

The aria-label, aria-labelledby, and aria-describedby attributes can be used with:

  • interactive elements like a (when the href attribute is present), audio and video (when the controls attribute is present), input (unless they are of type="hidden"), select, button, and textarea
  • elements that have a landmark role – either implicit (header, footer, main, nav, aside, section, and form) or explicitly set via the role attribute
  • elements that have an explicit widget role applied using the role attribute – there are 27 widget roles in ARIA 1.1, including dialog, slider, progressbar, and tooltip
  • iframe and img elements

If you use aria-label, aria-labelledby, or aria-describedby with any other elements (like div, span, p, blockquote, or strong etc.), they generally won’t work across all browser/assistive technology combinations.

One possibility here is to assign an explicit role. If that sounds good to you, I can run some simple tests using NVDA, or if you think you might want to try something else, create a simple version and I'll try it out again.

The second issue is a bit of a softer one. When using JAWS or Orca (avoiding the issue above) and tabbing through, it feels like the tabs are inconsistently labeled. You get three major genres of output: a heading (using the describedby) if the cell is markdown and a heading is present, the word "markdown" if the cell is markdown and a heading is not present, and code input if the cell is code. I think surfacing a heading if present seems like a good idea, but the inconsistency feels off. This might be something we can create a couple versions of and try out in testing, or do the async thing we were talking about.

Just for reference when reading this comment, here's an example of how the article element is set up currently:

<article aria-label="code input 1" class="jp-Cell jp-CodeCell jp-Notebook-cell" tabindex="0">
smythp commented 1 year ago

when adding the tabs we need to include a visual style too... i don't know a subtle way to do this. in fact, i don't know subtle.

Maybe let's just do a solid border for now? Just for testing things out. But our designers might have better ideas.

tonyfast commented 1 year ago

OK, tried it out, and the tab indexing works, which is pretty great. It currently moves from cell to cell, also landing on elements that typically are registered for tab indexing, like links.

thanks for digging into this and adding some better description to the task that i provided.

However, I do think we have some issues with the labeling. There are two main issues.

First, the labeling behavior is inconsistent among major screen readers. On JAWS, it seems to work as intended (seems like it speaks headings when headings are present, input and a number with code cells, and markdown when no headings are present, more on that below). It also behaves as expected using the (pretty terrible but worth testing with) Orca on Linux.

glad things worked until here. i was using orca and the accessibility tree for testing.

Using NVDA, however, when tabbing through cells, no output is presented at all, which is pretty disorienting when tabbing through. That is, when tabbing through the notebook using NVDA, it moves to the next cell, but doesn't speak anything aloud.

I can't say definitively why this behavior is occurring, but one strong possibility is that the labels are on an article element. From this page:

The aria-label and aria-labelledby attributes can be used to give an element an accessible name. The aria-describedby attribute can be used to give an element an accessible description. Not all elements can be given an accessible name and/or description though. The aria-label, aria-labelledby, and aria-describedby attributes can be used with:

  • interactive elements like a (when the href attribute is present), audio and video (when the controls attribute is present), input (unless they are of type="hidden"), select, button, and textarea
  • elements that have a landmark role – either implicit (header, footer, main, nav, aside, section, and form) or explicitly set via the role attribute
  • elements that have an explicit widget role applied using the role attribute – there are 27 widget roles in ARIA 1.1, including dialog, slider, progressbar, and tooltip
  • iframe and img elements

If you use aria-label, aria-labelledby, or aria-describedby with any other elements (like div, span, p, blockquote, or strong etc.), they generally won’t work across all browser/assistive technology combinations.

One possibility here is to assign an explicit role. If that sounds good to you, I can run some simple tests using NVDA, or if you think you might want to try something else, create a simple version and I'll try it out again.

Thanks for all the information here. I'm going to digest what you've said here and put into code.

The second issue is a bit of a softer one. When using JAWS or Orca (avoiding the issue above) and tabbing through, it feels like the tabs are inconsistently labeled. You get three major genres of output: a heading (using the describedby) if the cell is markdown and a heading is present, the word "markdown" if the cell is markdown and a heading is not present, and code input if the cell is code. I think surfacing a heading if present seems like a good idea, but the inconsistency feels off. This might be something we can create a couple versions of and try out in testing, or do the async thing we were talking about.

so the plan is to put all our of ideas or changes behind flags. i'm going to work on that immediately. this way when we have ideas we can configured them in jupyter_nbconvert_config file to turn them on or off. or A/B test.

the goal of this pull request will be to put all these concepts behind flags so we can test them out systematically.

Just for reference when reading this comment, here's an example of how the article element is set up currently:

<article aria-label="code input 1" class="jp-Cell jp-CodeCell jp-Notebook-cell" tabindex="0">
smythp commented 1 year ago

Tony, the flagging idea sounds like a good one. Thanks for putting together these prototypes, they're feeling like strong progress. :)

tonyfast commented 1 year ago

i've implemented the flags in the exporter. with these changes we can include any fixes we want as long as they are behind a flag. we can then populate a jupyter_nbconvert_config.py files to record the exporter inputs alongside an experiment. jupyter_nbconvert_config.py shows an example configuration.

tonyfast commented 1 year ago

adding accessible_pygments so we some accessible themes for pygments. it reduces soooooo many color contrast problems from axe.

we've add a few features in here. they are behind flags and we can accept them without much disruption. some of the changes make sense, but we should probably break out specific conversations about the features. a few issues i can see are:

tonyfast commented 1 year ago

everything is green so im not going to temp fate and merge here. this pull requests: