Share Cell Content with screenreaders

Jenneh commented 2 years ago

Problem and context

When the screenreader navigates to the top of a cell, It wasn’t immediately clear for the user what type of content they were about to read (ie. Markdown cell or code cell or cell output)They mostly guessed by context.

There is no feedback or concept of “cells” discovered by VoiceOver (though it seems like input and output text might get captured

Possible solutions

Label cells with what kind of content they contain: data cell, code cell.”

Acceptance criteria

Tasks to complete

isabela-pf commented 2 years ago

Oh, "Check Task 3 for In[1]: and Out[1]: text" was a note I left for exploring the problem. Sorry for the confusion!

As you noted, in sessions where screen readers were used the concept of "cells" in a notebook seemed basically nonexistent. Participants with past notebook knowledge were aware of cells from editable contexts, but didn't get any feedback about

what cells were
where cell breaks were
what kind of content they would be navigating into in a given cell
or if there was a relationship between the cells (usually for a cell input and its linked output)

But all of this info is communicated visually. It's missing when accessed via screen reader.

In one of our sessions, though, I thought I observed a case where a screen reader did capture the input-output relationship by reading the "In[1]:" and "Out[1]:" as proper HTML text. I left this note to remind me to follow up and review what happened since I believe we only heard it once across all our sessions.

Hope that helps! Let me know if you have more questions with my notes. 🌻

Side note: I think I found that the area where cell numbers and their input and output designation (when relevant) are visually always exists. Even when there is no relevant label for that cell visually, the region seems to exist with empty contents. I bring this up because it means we are capturing some kind of non-visual cell divide when the notebook is converted to HTML. There may be something we can put to good use there.

isabela-pf commented 2 years ago

Note to selves: loop this into test 2.

tonyfast commented 1 year ago

the tab to content configuration configures aria label on code cells. the aria label is cell {input_prompt}. this approach makes sense for notebooks that are executed in order. out of ordered and unexecuted notebook cells are going to need to be designed differently.

we are currently testing this in notebooks like https://iota-school.github.io/notebooks-for-all/exports/Imaging_Sky_Background_Estimation-tab-to-content-nav-high-contrast.html

is there more we need to address to close this issue?

smythp commented 1 year ago

Hi @tonyfast , we had a few conversations about the tabbing to cells not speaking anything aloud on NVDA. Right now, the basic structure of a tabbable cell looks like this:

<article aria-label="cell 1" class="jp-Cell -CodeCell jp-Notebook-cell" tabindex="0">

While this works fine in my testing on JAWS and Orca, it doesn't work with NVDA. Apparently aria labels on non-interactive or non-landmark elements are inconsistently implemented across screen readers.

I created a minimal example and tested whether giving an interactive role to the article element would make NVDA speak the label aloud. The short version is that this works. Here is a minimal example that reads the aria label aloud in NVDA:

<body>
  <article aria-readonly="true" role="textbox" aria-label="cell 1" class="jp-Cell -CodeCell jp-Notebook-cell" tabindex="0">
    <p>
      Some text
    </p>
  </article>
  <article aria-readonly="true" role="textbox" aria-label="cell 2" class="jp-Cell -CodeCell jp-Notebook-cell" tabindex="0">
    <p>
      Some text 2
    </p>
  </article>
</body>

When tabbing, NVDA will say the label, then say "edit," then say "read only."

Cell 1 edit read-only

I'm not sure if this is semantically ideal, but it does mean that the label gets registered with NVDA.

For completeness, here's a minimal example that doesn't get read aloud by NVDA (similar to the current Imaging_Sky_Background_Estimation-tab-to-content-nav-high-contrast.html:

<body>
  <article aria-label="cell 1" class="jp-Cell -CodeCell jp-Notebook-cell" tabindex="0">
    <p>
      Some text
    </p>
  </article>
  <article aria-label="cell 2" class="jp-Cell -CodeCell jp-Notebook-cell" tabindex="0">
    <p>
      Some text 2
    </p>
  </article>
</body>

Iota-School / notebooks-for-all