Dynamic strings should use i tags instead of em tags

[zepumph]

from https://github.com/phetsims/a11y-research/issues/66:

make the awesome highlighting of dynamic properties consistent for all sims. Use <i> tags instead of <em>, so that we can save <em> and <strong> for future a11y string specificity.

We also want to strip these dynamic i tags from the string files, and then add them in the sim code, like when filling in with StringUtils.

[jessegreenberg]

I remember that we had a discussion about this a while back and couldn't find notes. I remember @zepumph having a nice way to handle this, but can't remember exactly what was discussed. In the meantime, here is one potential way to do this:

• Add a query parameter to initialize-globals that indicates that sim is being run in a11y view, add this flag to query param list in sim-a11y-view.html.
• When in a11y view, iterate through {{SIM}}A11yStrings and wrap placeholder {{}} with <i></i>.
• When in a11y view, short circuit Accessibility.js functions setAccessibleDescription and setAccessibleLabel to setAccessibleDescriptionAsHTML and setAccessibleLabelAsHTML, so <i> tags are rendered in the a11y view.

PROS:

• All accessibility strings that are dynamic automatically get styled in a11y view.
• Requires no changes to simulation code.
• <i> tags do not have to be added to a11y strings in {{SIM}}A11yStrings.js/json files.

CONS:

• Maybe some strings that are not meant to be dynamic for a11y purposes will get styled. For instance the "Grab {{balloonLabel}}" pattern in BASE is not really a dynamic string, but it is assembled with StringUtils.fillIn() for i18n.

[jessegreenberg]

To get around the CONS, we could either

• Structure the a11y strings such that static strings do not use string patterns. Again, this is likely not as good when we decided to support i18n and a11y together, could complicate the implementation.
• Add data to the string JSON files that indicate whether strings are dynamic or static. If added to string pattern, entire sentence would be styled as dynamic. If added to variable, the string would be styled as dynamic everywhere (even if not used in cases where it changes).

[terracoda]

@jessegreenberg, I think it is ok that examples like "Grab {{balloonLabel}}"} would get styled as dynamic content. We could, however, easily override the style a nested tag inside buttons, inputs and other controls should we choose to do so with something like this:

button i {
  border-bottom: none; /*(or text-decoration none)*/
  background-color: inherit; /*(or blue)*/
}

[jessegreenberg]

Good point @terracoda, that would work for the case of BASE.

[zepumph]

Could we get a consensus on how to proceed with this issue?

[jessegreenberg]

@zepumph do you have any thoughts about https://github.com/phetsims/a11y-research/issues/67#issuecomment-358797815? I would be happy to proceed with that unless you have concerns or a better recommendation.

[zepumph]

Requires no changes to simulation code.

I'm still in brainstorming mode. To me this doesn't necessarily seem like a pro to me. Handling this completely in the a11y view is a post processing step, which we will have to maintain as things change in the sim.

What would be pros and cons of automatically surrounding a11y strings using StringUtils.fillIn() with <i> tags?

PROS Integrated into the simulation so we don't forget about it as an edge case CONS Would we have to change each call site of fillin? Maybe we can get around this by adding an a11y option to StringUtils.fillin.

I'm not sure about this, but want to discuss further. Marking for a11y dev meeting.

[jessegreenberg]

Discussed in a11y dev meeting on 2/21/18.

We like the proposal in https://github.com/phetsims/a11y-research/issues/67#issuecomment-358797815, but still have questions about the second bullet point.

Right now, a11y strings are in separate .js/.json files (they should all be converted to .json in the future), and are added to the phet namespace. From here we can modify string values in initializeAccessibility() in Sim.js. But when a11y strings are moved to translatable string files, we won't be able to access them. We can't think of a way around this. We don't want to proceed because we don't want to paint ourselves into a corner later.

We explored manipulating the string plugin to modify values, but ran into problems because the built sim hardcodes the strings in.

[jessegreenberg]

https://github.com/phetsims/a11y-research/issues/67#issuecomment-367457505 is very complicated. @zepumph suggested that we wrap dynamic a11y descriptions with <i> tags always so we don't have to check for an "a11y-view" mode in the built version of the sim. This would require that we remove functions setAccessibleLabel and setAccessibleDescription, and use setAccessibleLabelAsHTML and setAccessibleDescriptionAsHTML everywhere. Always using these functions could degrade performance so we want to do some benchmark tests before proceeding. We also want to check with @terracoda that we can have <i> tags always surrounding dynamic strings in the PDOM, even when the sim is not running in a11y-view mode.

To add the <i> tags in the PDOM, we could surround dynamic strings with the tags in the string files directly. Another way to do this could be to modify strings from the string pluggin in the load function. There may still be a better way to do this.

[jessegreenberg]

Another idea worth investigating - can we do this with MutationObserver alone by comparing diffs of mutations?

[jessegreenberg]

Discussed in a11y meeting 3/2/18 -

@terracoda suggested using a span with a class because which will not have any unwanted semantics. @jhung agrees.

@terracoda suggested testing behavior of adding span, i, em, or b tags.

[jessegreenberg]

@zepumph said that a solution in the a11y-view is less maintainable because because it is separate from the sim.

[jessegreenberg]

We will discuss next steps in an a11y dev meeting.

[jhung]

@jessegreenberg, @terracoda , and @zepumph - I spoke with @klown and here's what he thought: "could also use a selector based on a role, if that's applicable, or even an aria-* attribute.

For the record, the a11y mapping for em is not semantic at all. It's mapped to a style: https://w3c.github.io/html-aam/#el-em (I wonder why…). The same with strong. https://w3c.github.io/html-aam/#el-strong

So, it looks like using em and strong semantically is lost at the level of the a11y tree. Hence, don't use them. The decision you and the PhET people made [i.e. using <span>], jhung, was correct."

[terracoda]

@jhung and @klown, where does it say in the specification that the "a11y mapping for em is not semantic at all"? I'm not sure how to read the specification ;-)

I would also argue that style itself does communicate have meaning, especially in terms of a user's experience. For example, headings are bigger and bolder to communicate their importance.

Maybe a better way to frame the problem is that we do not want to miss-communicate with style. Here's a classic miss-communication example <h3 class="main-heading">My Page's Main Heading</h3>.

In the case of the Sims Ohm's Law and Resistance in a Wire, I have heard JAWS read out content within strong tags in a different tone of voice.

I think that we should try to avoid a change in tone of voice when dynamic strings are read out by AT. If a span with a class ensures no change of tone, great.

If we only put the needed tags (whichever one works best) in the A11y View's PDOM copy, then we would not be messing with the Sim User Experience at all. Is that what you were getting at @jessegreenberg and @zepumph . If that is a possible route to consider, it might be might be the safest.

[jhung]

@terracoda my comment was quoting what @klown said (aka. Joseph Scheuhammer at the IDRC).

[terracoda]

@jhung, I realize that you were quoting @klown.

@klown, perhaps you could explain where the specification says that the "a11y mapping for em is not semantic at all."

I am just trying to understand the specification better. If you can help, thanks.

[jessegreenberg]

If we only put the needed tags (whichever one works best) in the A11y View's PDOM copy, then we would not be messing with the Sim User Experience at all.

We looked into that a little bit, but only adding tags in the a11y view would be more difficult to implement.

[jhung]

@terracoda I spoke with @klown to get some clarification on his comment. He said that the <em> element appears in the accessibility tree but there's no node for it using the accessibility API. So in effect the accessibility API "loses" the semantics of <em> even though it's present in the tree.

Klown was surprised that this was the behaviour because there should be semantics, but the API doesn't give it.

[terracoda]

@jhung thanks.

I agree that it is strange that the "semantics" for em and strong are lost in the accessibility tree some where-some how!

Getting back to our actual issue, we are trying to create the simplest CSS hook through which we can easily and consistently style dynamic parameters within dynamic descriptions when represented in a sim's A11y View.

Even though it is a bit more code, perhaps the surest non-semantic option is a span element with a class attribute. The span element will never have any machine-readable semantics, and the class attribute will provide human-readable semantics to sim developers.

<span class="parameter">some dynamic content</span>

[zepumph]

the "semantics" for em and strong are lost in the accessibility tree some where-some how

From reading the above comments, I don't know if this is because of the spec, or just how it is; nor do I know how this may change in the future.

The above commit is beside the point though, because it has been settled that a span is the best option for provided zero semantics to an element for syntax highlighting.

Let's proceed with the following directive:

Dynamic strings should be wrapped in a classed span to provide opportunity for syntax highlighting in the a11y view.

Please correct the above bold if that is an incorrect focus for the rest of the issue.

Here are the following questions/ideas to figure out:

• What is the scope of this change? Is this only valuable in the a11y view, or are there other places where wiring into the "dynamic" pieces of the a11y strings (via the classed span) may be valuable (perhaps phet-io?)?
• The above question will influence how we go about implementation. Do we manipulate the strings in the sim, or post-process add the span in the a11y view.

[zepumph]

I'm leaning more and more toward just having this for postprocessing on the a11y view, although it seems like the implementation may be more complicated.

[terracoda]

@zepumph, post processing on the ally view is the ideal situation in my opinion. This would keep extraneous non-semantic elements out of the PDOM and sim code in general.

We do not yet know all the use cases, nor all the useful applicable aspects of our descriptive content, so I hate to hard code something in where it should not be.

With little experience with PhET-iO, I can't comment too much on that. For accessible sims that are available on PhET-iO, it seems we would need to make all descriptions (static, dynamic, and interactive alerts) available in some way, at least to verify any changes to the visual sim would remain in synch with changes to in the non-visual representation of the sim.

[terracoda]

@zepumph, you are manipulating the strings somewhere along line. Are there multiple places along the line that the strings are being manipulated, or just one place?

[zepumph]

I want to make sure we are all on the same page about the current state of this issue, the problem at hand, and the potential solution we are looking for.

Currently we are adding css to <em> tags so that they show up green and underlined. These em tags are just in the a11y strings for Ohms Law and Resistance In A Wire. To me, there seem to be a few problems with this option:

1. We decided early on that <em> holds semantic information, so we don't want a generic solution that involves sticking em tags all over pDOM strings that don't semantically need/desire that surrounding tag.
2. We just want this for visual style to the pDOM in the a11y view tool. It seems to be pretty unanimous that having a tag surround dynamic pieces of strings is not useful for anything but the a11y view.

So that being said, we have two sims, RIAW and OL, that just happen to use em tags around their dynamic strings for emphasis on those strings, but that is in no way a generic rule about how we want to create dynamic strings.

When we separate these two pieces out into the very different things that they are, we see independent features of the sim. From here on let's ignore that fact that these sims use the em tags, unless I am wrong in thinking that "they were added to the sims for semantics, and not for a11y styling."

We also decided that in the a11y view, we would rather have <span class="dynamic"> surround the dynamic strings instead of em.

So we want a11y view styling for dynamic strings, and we don't want to interfere with the actual strings that will be translatable and stored in the sim. I am quite stumped as to how to get this. @jessegreenberg, didn't you look into the MutationObserver to try to get this accomplished. To me it seems very difficult to detect this. How would we detect string changes differently from large sections of the pDOM being created or destroyed, or if a Node was redrawn and had updated text.

Backtracking a bit to talk about manipulation of strings on the sim side (sorry for the longest github message), the easiest solution from an engineering perspective would be to just manually add the span to surround any placeholder/dynamic (I know those aren't always synonymous) string. This is not great for the above mentioned reasons ((a) now these spans are always in the pDOM, (b) they are editable by translators, which is a bit scary). We also talked about potentially adapting that way that we fillIn placeholders to automatically add these strings in (still the problem with these spans always being in the pDOM).

All in all, I don't love either of the solutions in the penultimate paragraph, nor do I love the idea of messing with the MutationObserver. I'm feeling stuck, out of ideas. Does anyone else have any thoughts or suggestions? How important is this anyways. I really enjoy the highlighting in RIAW and OL, it's really nice. Note that this issue is 5 months long with no end in sight.

[zepumph]

Talked about during the IDRC meeting today. We discussed the following:

• It will not work to implement this just in the a11y view, it will have to come from tags in the strings
• since they will be in the strings in the pDOM, we like using <i> instead of span, because it is much cleaner, shorter and simpler.
• It isn't a great solution to automatically add the tags around curly braces because some of the pattern strings are still "static" in terms of how they change in the pDOM
• What we want to highlight is a fluid concept. @jessegreenberg says that at any frame scenery can potentially redraw the entire pDOM, so isn't it all dynamic? We decided that it is important to highlight "key dynamic values" that help guide eyes to understanding how the pDOM is changing as you interact with the sim. Basically now this is another design step.
• We don't want it to be an overarching, mandatory convention used by the phet team, because we sense that it may not be necessary. @emily-phet thinks we should focus on a few sims, ones that we will share with others or teach phet a11y to.
• @terracoda recommended using the i tags to highlight regions instead of just phrases.

[terracoda]

Clarification, @terracoda recommended styling the full dynamic descriptions or regions. That would not involve i-tags :-)

[zepumph]

Thanks for speaking up, I must have missed something towards the end there. Could you clarify? How would we mark which regions were fully dynamic (and needed the styling) if not i tags?

[terracoda]

@jessegreenberg, brought up a great point in the meeting that what I was representing as "dynamic descriptions" was actually the dynamic parameters within "dynamic descriptions". I would like to explore the possibility of something that can allow us to style the dynamic descriptions at a higher level.

[terracoda]

@zepumph, from a front-end of point of view styling HTML content is a straight forward thing to do - tricky to get right sometimes, but the concept is simple.

What was made really clear today is that we do not yet have an easy way to create the styling hooks.

I'd like to explore again, your idea of placing a flag of some kind further up in the code.

Something that might allow use to place classes in appropriate places in the PDOM copy.

It might be easier to identify static content. I think you mentioned this in the meeting. I'd like to discuss that in more detail.

[emily-phet]

hey folks - I don't the benefits of investigating this further are worth the time it will take, right now. Let's focus on team integration efforts (getting infrastructure and processes in place to support full team implementation of a11y) and further development of a11y features in sims. We can come back to this later, and decide specifically what the benefit of highlighting subsets of the text are, and then we can dig into how to best do it.

[terracoda]

I agree with @emily-phet that further work on the look of the A11y View can wait.

While I think we can still improve the general visual Iook of this tool, the styling of dynamic string parameters is not the way forward.

[zepumph]

Sounds good!

[jessegreenberg]

Removing meeting:a11y label since this was deferred.