How to do accessible scientific notation

jessegreenberg commented 6 years ago

Part of the larger question "How will PhET do accessible math?", but we will need scientific notation in accessible coulombs-law and gavity-force-lab-basics. How is this typically done?

@arouinfar said:

Ideally, something like -3.67E-2 would be read as, "negative three point six seven times ten to the minus two."

Perhaps AT will recognize this notation automatically? Or maybe they will say "negative three point six seven E dash 2".

@terracoda do you have any thoughts? Putting on schedule for tomorrows a11y meeting.

arouinfar commented 6 years ago

Just to clarify, I used the E notation because I was typing a comment in a Google doc. Sims will not use the E notation. Instead, -3.67E-2 would be -3.67 10^-2 where is the appropriate multiplication symbol (not just an x) and -2 is a superscript.

Here's an example, though with a different value.

It looks like the string might be using the letter X to represent the multiplication symbol. We should, however, be using the proper mathematical symbol, as in Area Model Multiplication.

jessegreenberg commented 6 years ago

Thanks for clarifying @arouinfar, JavaScript also uses "E" notation for scientific notation so I thought it might extend to AT, but I could be wrong.

It looks like the string might be using the letter X to represent the multiplication symbol. We should, however, be using the proper mathematical symbol, as in Area Model Multiplication.

Sounds good, ~~Ill make an issue in ~~coulombs-law.~~ inverse-square-law-common.~~ EDIT: @arouinfar beat me to it.

arouinfar commented 6 years ago

I opened the above issue to make sure we are using the appropriate multiplication symbol in ISLC sims.

arouinfar commented 6 years ago

Looks like we were commenting simultaneously @jessegreenberg!

Thanks for clarifying @arouinfar, JavaScript also uses "E" notation for scientific notation so I thought it might extend to AT, but I could be wrong.

I am not sure about AT norms, but classroom norms for scientific notation would be to use the x10^ notation.

arouinfar commented 6 years ago

@arouinfar said:

Ideally, something like -3.67E-2 would be read as, "negative three point six seven times ten to the minus two."

Actually, I'm not sure the best way to describe a negative exponent. In science, we often use "minus" and "negative" interchangeably in exponents. We should probably be consistent and use "negative" but that detail can be ironed out later.

terracoda commented 6 years ago

Doing math content will be a new experience for me. Sure @jessegreenberg we can put it on the A11y meeting schedule.

terracoda commented 6 years ago

I am just going to post a recent research article which likely has some good resources in the Works Cited section, New Accessibility Features in MathJax

emily-phet commented 5 years ago

Is there a straightforward answer to the question - "can we currently use MathML for mathematical notation"? Or will this require some investigation?

jessegreenberg commented 5 years ago

PhET cannot use MathML currently because we are transitioning to claim support for JAWS with Chrome (https://github.com/phetsims/QA/issues/217), and Chrome does not support MathML.

Firefox and Safari both support MathML according to https://caniuse.com/#search=mathml, so if it is important to use MathML then maybe we could reconsider the decision to transition to Chrome.

emily-phet commented 5 years ago

@jesse - thanks for the reminder on that. It was probably brought up before.

For this issue - since MathML is not an option, we just need to determine how we want the numbers to be read out - e.g., "3.14 times 10 to the minus 4 units" or something else. Are there ideas for where we might find a consensus answer to this question?

jessegreenberg commented 5 years ago

Could we start with MathJax or MathML and hear how they read out scientific notation on the platforms where they work? Then we could add something that sounds identical without using those tools directly.

terracoda commented 5 years ago

@emily-phet, I can also ask our consultants tomorrow and Friday this week. I'll run the phrases I have created for scientific notation by them.

emily-phet commented 5 years ago

@jessegreenberg - let's definitely check out how MathML would read this out and get feedback on this wording.

jessegreenberg commented 5 years ago

Sounds good - One thing I learned is that for MathML to work with NVDA it requires a third party plugin MathPlayer https://www.dessci.com/en/products/mathplayer/

jessegreenberg commented 5 years ago

OK, so here is MathML using to test scientific notation:

  <math xmlns="http://www.w3.org/1998/Math/MathML">
    <mn>7</mn>
    <mo>&times;</mo>
    <msup>
      <mn>10</mn>
      <mrow>
        <mo>&minus;</mo>
        <mn>4</mn>
      </mrow>
    </msup>
  </math>

@emily-phet @terracoda NVDA says: "Seven times ten to the negative four power"

After installing MathPlayer, NVDA says this in Chrome too! This is surprising because it looks bad in Chrome: captuire

And chrome reports the accessibility tree as empty: capture

But maybe it is possible to use MathML in chrome anyway? Or maybe there is just very minimal a11y support in Chrome.

emily-phet commented 5 years ago

Well - that sounds just fine to me. Let's go with this wording. "X.X times ten to the negative Y power". Thoughts on any implementation plan changes with this new possible Chrome support of MathML?

jessegreenberg commented 5 years ago

JAWS reads it out as "Seven times ten to the power of negative four, math content" in Chrome. And when I press "enter" on the content, a "Math Viewer" pops up and I can read through individual items in the expression.

capture

I think we should try to use MathML in PhET as much as possible since it seems to be working.

But @emily-phet MathML requires Windows users install this third party software. Is that acceptable?

emily-phet commented 5 years ago

Hmm - I'm inclined to go with using MathML as well. From my understanding, this is the standard the broader a11y community is moving towards. It's also likely that a student in a math or science class using a screen reader would already be using MathML, or could benefit from using it.

terracoda commented 5 years ago

@jessegreenberg, can I try reading your example page with VoiceOver in Safari?

Maybe I should try your mathml code above in an HTML page?

terracoda commented 5 years ago

With out anything special and an HTML5 doctype, I got something a little less nice in VoiceOver and Safari

7 times 10 with superscript negative 4, with 3 items, math

Oops, corrected math content to 7 times 10.

terracoda commented 5 years ago

The phrase is still recognized as math and the content provided is correct. "superscript negative 4" is perhaps less ideal than "to the power of negative 4".

jessegreenberg commented 5 years ago

https://github.com/phetsims/a11y-research/issues/105#issuecomment-437216862

I totally agree @emily-phet.

https://github.com/phetsims/a11y-research/issues/105#issuecomment-437313988

Please and thank you @terracoda! Good to know that MathML is recognized, but unfortunate that it doesn't sound so nice in VoiceOver.

jessegreenberg commented 5 years ago

There are a few things about using MathML for accessible math that would be good to talk about at the next design/dev shorts meeting and it would be good to get input from both developers and designers.

Negatives: 1) Scenery's a11y support wasn't really designed for markup like MathML. Creating the structure in PhET code isn't very pretty:

Node code:

```js var mathNode = new Node( { tagName: 'math', accessibleNamespace: 'http://www.w3.org/1998/Math/MathML', children: [ new Node( { tagName: 'mn', innerContent: 7 } ), new Node( { tagName: 'mo', innerContent: '×'; } ), new Node( { tagName: 'msup', children: [ new Node( { tagName: 'mn', innerContent: 10 } ), new Node( { tagName: 'mrow', children: [ new Node( { tagName: 'mo', innerContent: 4 } ), new Node( { tagName: 'mn', innerContent: 4 } ) ] } ) ] } ) ] } ); ```

We can certainly come up with ways to make this better.

2) The output on VoiceOver is not so nice, see https://github.com/phetsims/a11y-research/issues/105#issuecomment-437317717 3) MathML requires third party software to be usable with NVDA on Windows. 4) Even outside of scenery, MathML can be difficult to write, see the complex examples in http://gorupec.awardspace.com/mathml.xhtml

Positives: 1) MathML is a standard being used in the broader a11y community, see https://github.com/phetsims/a11y-research/issues/105#issuecomment-437216862 2) We don't have to reinvent the wheel for accessible math. We will never have to decide "How to do accessible fractions" or "How to do accessible scientific notation". 3) Reduces number of strings we create, and makes i18n easier. 4) MathViewer and JAWS have ways user can explore math expressions in detail, which would be clumsier if we spell things out like "5 times 10 to the negative fourth power".

Overall, I think the positives outweigh the negatives, but I want to make sure we are all in agreement.

jessegreenberg commented 5 years ago

I also think that whatever we decide we aren't locked into one way or the other. If we try MathML and decide it won't work out we could switch to rolling without it and vice versa. Especially since the a11y sims coming up don't involve a lot of math expressions.

terracoda commented 5 years ago

@emily-phet, I am not sure we know enough about accessible math solutions to fully embrace MathML.

Would it be possible to go with a spoken text-based alternative until we know more? I think we can deliver a more consistent experience for the simple math we need at this point.

We could set aside some resources to investigate and document possible options that could work for PhET sims.

emily-phet commented 5 years ago

@terracoda I'm inclined to go with MathML for now. As @jessegreenberg mentioned, we don't have to commit to it for life at this point. Because the math we are expressing is so simple, I think it would be good to at least for now be consistent with what the broader community is moving towards (and advocating for). Users familiar with MathML will have additional affordance we won't provide without MathML, and we'll be able to say we're using MathML until we encounter a specific challenge that makes us choose a different path. Right now, we'd be choosing a different path largely in anticipation of a potential breaking problem - which feels less compelling to me.

We certainly could adopt to the extent it works for us now, knowing that as we move towards more complex math expressions (e.g., Plinko!) we'll need to investigate some more each step along the way.

Of course, we can talk more in real-time on Tuesday!

terracoda commented 5 years ago

@emily-phet, I actually agree that MathML is good direction. I missed @jessegreenberg's comment about us being able to change https://github.com/phetsims/a11y-research/issues/105#issuecomment-437533296 and was concerned a little about VoiceOver's output.

The positives that @jessegreenberg laid out are strong ones.

jessegreenberg commented 5 years ago

One question @terracoda and @mbarlow12 brought up is that in the GFL sims, the scientific notation may need to be announced in an aria-live alert. PhET's alert system currently doesn't support markup. And we are not sure if aria-live supports inner content with MathML.

jessegreenberg commented 5 years ago

We will need some of our alerts to include math content. We should test whether or not MathML can be read in an aria-live alert. If it can, than all math can use MathML everywhere. If it works, we will also need to explore how to send our own MathML to the utteranceQueue, and embed it into sentences.

If those things won't work, we will need to send strings representing the math for alerts, and use MathML for the actual Math content that persists in the DOM.

@emily-phet said that we should proceed with MathML because scientific notation is a simple test case for us to use MathML, and we will certainly want this in other sims in the future. It would be confusing for us (and users) if math is inconsistently represented with MathML in our sims.

We will proceed with MathML now, @mbarlow12 will explore this in his work in GFL. Good luck!

jessegreenberg commented 5 years ago

@mbarlow12 said he could work on this this week and will check back in with the team next week about how it is going with MathML.

jessegreenberg commented 5 years ago

While looking into https://github.com/phetsims/scenery/issues/870 found that MathML doesn't support the hidden attribute and others. So I guess to make invisible from a screen reader the MathML element will need to be wrapped by an HTML element which can be hidden.

terracoda commented 5 years ago

@jessegreenberg, does the aria equivalent, aria-hidden="true" or aria-hidden="false" work any better?

I would think that there are many HTML attributes that are no supported by MathML as they are separate mark-up languages.

jessegreenberg commented 5 years ago

Good point @terracoda. We synchronize hidden with scene graph visibility in scenery which is how I found it wasn't getting hidden by scenery. Ill try aria-hidden and see if that works for MathML.

jessegreenberg commented 5 years ago

aria-hidden works on MathML in FF and Chrome!

phetsims / a11y-research

How to do accessible scientific notation #105