Consider addition of property(ies) to expose braille string

[joanmarie]

A couple of use cases have come up in which it would be desirable to give assistive technologies braille strings which they could display as-is:

• Abbreviated/localized role descriptions
• Nemeth rendering for math content

With respect to the former, we created aria-roledescription so that authors could customize what is presented to the user by assistive technologies. A side effect/consequence of this feature is that it stomps on the screen reader's ability to provide abbreviated role names. For instance, if the ARIA role is button, a screen reader could abbreviate that as "btn" in order to be able to show more information at once on the refreshable braille display. But if an element has an aria-roledescription value of "Pizza slice" (to borrow an ETS example), that value overrides what the screen reader should display -- and occupies potentially more braille cells than may be desired. It would be handy if an author could specify "ps" or "slice" or whatever as the braille role name.

With respect to the latter, there are tools which can produce a spoken representation of math expressions. Assistive technologies which can take advantage of such a tool can pass the string along to be spoken. But that spoken representation is likely not what should be displayed in braille. Hypothetically, such a tool could also provide a Nemeth string which should be similarly passed along unmodified. But there's no standard means to expose this information to assistive technologies.

[jnurthen]

This feels like a very similar problem to ruby. Maybe we could use whatever solution we come up with for that too.

[arnog]

For a given expression, authoring tools might want to produce multiple representations of that expression, for example a spoken representation, a Nemeth Braille representation, a UEB Braille representation and a (plain) braille representation (particularly useful for users that know Braille but may still be learning Nemeth Braille). Having a mechanism to author this would be very useful.

[pkra]

@joanmarie @jnurthen could we put this on the WG agenda some time?

[pkra]

Some initial ideas: https://github.com/w3c/aria/wiki/*-braille-properties-for-exposing-custom-Braille-descriptions

[garconvacher]

I dream of an aria-label-braille! Example with International Phonetic Alphabet:

<p>[<span aria-label-braille="⠃⠢⠸⠝⠡⠒⠝⠢">bəˈnɑːnə</span>] est la prononciation du mot "banane" en anglais.</p>

Currently, we use:

<span class="sr-only">⠃⠢⠸⠝⠡⠒⠝⠢</span><span aria-hidden="true">bəˈnɑːnə</span> 

Unicode braille output on braille displays is fine on every platform with the most popular screen readers.

(Almost) Off topic, @pkra, my colleague work on a JS script to convert MathML on braille on the fly. French math braille code is done. Nemeth and UEB are WIP.

[cookiecrook]

I disagree with the use case of Nemeth. Nemeth for for MathML and LaTeX is already supported by Apple's braille implementation, which provides a better interaction (it includes sub-formula exploration) than a static braille string can provide.

This proposal also offloads the effort of complicated braille translation intricacies to web authors, who are unlikely to understand the problems. For the Math issue, the better solution is to evangelize to Google, who removed their MathML implementation several years ago.

How could a web page author possibly know that a specific SR user wants to always read Math strings in computer braille (not Nemeth), or always expects standard strings to be rendered in 8-dot, or even wants English punctuation to be rendered in German braille. These are all real use cases that we support, and I have names in mind of real people that use it this way.

FWIW, I think there may be some legitimate use cases, but I don't think Math is a good one. Just use any of the standardized Math formats.

[cookiecrook]

@garconvacher wrote:

my colleague work on a JS script to convert MathML on braille on the fly. French math braille code is done. Nemeth and UEB are WIP.

This is a great example. even with these translations, how would a page author know the user prefers French Math Braille over Nemeth? I'm not convinced "Braille for Math" is a web author problem. As long as the author can represent Math in a standard format, the rendering engine and screen reader should do the rest.

[cookiecrook]

Would it be possible to solve some of the use cases by indicating a preferred braille table? e.g. aria-brailletable="UEB"... I'm not proposing this. I'm just trying to understand the use cases better.

[joanmarie]

@cookiecrook This feature is not limited to presentation of formulas. That said...

What about authors who choose to not use MathML? Maybe an author created a custom web application that provides its own navigation and exploration, using live regions for the spoken presentation. We wouldn't want screen readers to display the spoken presentation, and the screen readers wouldn't have the underlying semantics in this case to put it together themselves.

I can imagine a similar app for exploring musical scores. What are we going to do about music braille?

I believe ETS might also have uses for being able to customize the braille string.

What I think we want to accomplish in this feature is to make it possible for authors with both a need and the requisite braille knowledge to customize the braille presentation.

[cookiecrook]

@joanmarie wrote:

What about authors who choose to not use MathML?

Perhaps the Math problem would be better generalized as aria-math="[linearized LaTeX string]"

Maybe an author created a custom web application that provides its own navigation and exploration, using live regions for the spoken presentation. We wouldn't want screen readers to display the spoken presentation, and the screen readers wouldn't have the underlying semantics in this case to put it together themselves.

That's pretty vague. Can you connect the dots (pun intended 😉) with a few more specific cases?

I can imagine a similar app for exploring musical scores. What are we going to do about music braille?

I have heard of some Music markup formats. Are there there any that are generally linearizable like LaTeX is to math? If so, perhaps aria-musicnotation="…"

I believe ETS might also have uses for being able to customize the braille string.

• @mhakkinen, @jasonjgw

What I think we want to accomplish in this feature is to make it possible for authors with both a need and the requisite braille knowledge to customize the braille presentation.

A web author will likely never know the user's braille-specific settings/preferences, so even with requisite deep braille knowledge, this will be challenging.

[cookiecrook]

If the braille specific role description feature is added, it should disallow uses of known role types... E.g. The braille representation of the "button" role in VoiceOver is "btn", so a web author should probably not try to make this "bn" or "b"…

If the braille role description abbreviation is added, I think there should be a few RFC-2119 statements in the prose (loosely phrased here of course):

1. ~ Authors should not use this for known role types... Only use this if there is a custom role description also provided. To use the ETS example:

   [role=button [roledescription="slice" [brailleroledescription="slc"]]]

2. ~User agents should ignore brailleroledescription if roledescription is not provided, or if roledescription matches the default role description for the role.

Maybe more?

[sinabahram]

Proposing a braille table would go a heck of a long way to disambiguating some of the various issues that can come up. May I suggest, though, that a default braille table be assumed since that’s easily getting into deep Braille knowledge.

[arnog]

As an example, we're the author of a web app which is a math editor (see mathlive.io) (technically, it’s an open source Javascript library which is used in a number of third party applications on the web). We have built our own UI to enter and edit math formulas. Currently there’s no web standard to output to Braille, which I’d be happy to do if it was possible. In this scenario, I would offer users the possibility to customize and choose which format they want to use.

Having a standard to output Braille would not be a solution that content authors would be expected to use, but it would fill a gap for authoring tools to create accessible content.

[joanmarie]

@joanmarie wrote:

[...]

Maybe an author created a custom web application that provides its own navigation and exploration, using live regions for the spoken presentation. We wouldn't want screen readers to display the spoken presentation, and the screen readers wouldn't have the underlying semantics in this case to put it together themselves.

That's pretty vague. Can you connect the dots (pun intended ) with a few more specific cases?

Here's an interesting write-up: https://www.peterkrautzberger.org/0209/. See the CSS and SVG equation.

I really like how the navigation and voicing were implemented. If accessible equation editors were to all follow this lead in terms of the implementation, end users would wind up with an interoperable user experience regardless of platform. But the braille experience would be lacking.

I get the perspective some hold that everyone should just use MathML. And regarding evangelizing to Google, Igalia is working on implementing MathML in Chromium as we speak. BUT even after MathML is in Chromium, it doesn't mean authors are going to automatically start using it -- just like they didn't stop using divs and spans and JavaScript to create a UI experience which some would say could have been accomplished by using native HTML widgets. We cannot control what authors do; we can merely try to provide them with tools to make what they create more accessible and with guidance to do it right. I think this feature is one of the tools.

[...]

A web author will likely never know the user's braille-specific settings/preferences, so even with requisite deep braille knowledge, this will be challenging.

Agreed. That's the guidance side, which will be addressed in Authoring Practices.

[cookiecrook]

@arnog wrote:

As an example, we're the author of a web app which is a math editor (see mathlive.io) (technically, it’s an open source Javascript library which is used in a number of third party applications on the web).

I acknowledge this is a good, if rare, use case.

Having a standard to output Braille would not be a solution that content authors would be expected to use, but it would fill a gap for authoring tools to create accessible content.

This is my worry. Well meaning but misinformed web authors that don't understand braille and don't need to use this attribute may discover it and think this is a requirement for braille support. I don't want an author to auto-translate all their strings into Grade 1 for example. It could result in a much worse experience for most braille users, as it could not take into account preferred translation tables, braille verbosity settings, etc.

At a minimum any spec covering this should be peppered with warnings saying, "No really. You don't need to use this. Please don't unless you know what you are doing."

Alternatively, consider a separate Braille-on-the-Web incubation effort in WICG. This might result in some new ARIA attributes, but it could be a focused effort that defines the use cases, discusses potential solutions, and prototypes experimental implementations. If it really ends up this simple (a braille unicode string and brl role desc) that's fine, but it should be considered holistically.

[jasonjgw]

I support James's options. I think the use cases (chemistry notation, for instance, which is not fully addressed by MathML, music notation, and overcoming implementation issues in MathML to braille translation) for providing the author with flexibility to specify braille directly, are compelling. There are also detailed issues that haven't been discussed - for example, what happens if the user issues navigation commands to move character by character or word by word through text for which the author has supplied a braille translation? Is there a better way to support custom navigation (e.g., by subexpression in mathematics or chemistry) than by creating elements with application role and capturing keyboard events? The latter approach creates usability complexity in some screen readers, as it switches the screen reader into focus mode rather than browsing mode within the application role - and, semantically, there is not a genuine "application" involved.

Of course, one might argue that these are merely implementation problems; but we also need to be sure the standard enables the implementations to provide a suitable UI which, for example, doesn't require the user to switch modes unnecessarily in Linux and Windows-based screen readers. If we are confident that the proposals currently under discussion are a solid foundation that can be refined later, I'm in support of them. If not, we'll need to opt for a more elaborate design effort.

[joanmarie]

From discussion during TPAC:

• The braille value can be one of two things: 1) A non-braille, localized string (e.g. "pln"). 2) An already translated string (e.g. "⠏⠇⠝").
• The non-braille, localized string gives the user's AT the ability to translate the abbreviation via the user's preferred translation table. The braille string would be taken as-is and passed along to the user.
• We will use prose in the spec to explain the two different types of strings which are possible. BUT we will only include the non-braille, localized string in the example. This will (hopefully) maximize the likelihood that users will get the string in the format they want and expect from the average author.

[pkra]

Thanks for the update. I'll make the changes as discussed later this week.

[pkra]

While we still have several open issues to refine things further (see the braille project board), aria-brailleLabel is in the main branch (alongside aria-brailleRoleDescription) and aimed to be included in ARIA 1.3.

It seems to me this issue has served its purpose could be closed.

[pkra]

While we still have several open issues to refine things further (see the braille project board), aria-brailleLabel is in the main branch (alongside aria-brailleRoleDescription) and aimed to be included in ARIA 1.3.

It seems to me this issue has served its purpose could be closed.

Now that those PRs have all been merged and nobody objected, I think we can close this issue and track future related issues via the Braille project board.

Please feel free to reopen.