search

dotnet / machinelearning-modelbuilder

Simple UI tool to build custom machine learning models.
Creative Commons Attribution 4.0 International
265 stars 56 forks source link

A11y_ML .NET Model Builder_ Image Classification-Data _ Usability: In narrator scan mode when user navigates to "example folder structure" using up/down arrow keys it would be better if narrator reads the levels along with reading the content #1154

Closed SaisreeDandamudi closed 3 years ago

SaisreeDandamudi commented 3 years ago

“Check out Accessibility Insights! - Identify accessibility bugs before check-in and make bug fixing faster and easier.”

GitHubTags:

A11yUsable; #A11yTCS; #Win32; #DesktopApp; #Feature-ML.NETModelBuilder-Win32-Nov2020; #A11ySev4; #Benchmark; #Visual Studio; #Screen Reader; #Narartor; #NVDA;

Environment Details:

Microsoft Visual Studio 2019 Preview Version 16.8.0 Preview 6.0

Screen Readers:

Narrator NVDA

Repro Steps:

1.Start Screen Reader 2.Open Visual Studio Enterprise 2019 Preview

  1. Create a new .Net Core Console App (C#)
  2. Tab to "Solution Explorer" and right click on the project (Shift + F10) and then from the expanded list Select "Add" and then from the list items select "Machine learning"
  3. Tab till "Image Classification" scenario and select the scenario
  4. Select any of the environment and tab till "next step" button and press enter
  5. Enable scan mode and navigate till example folder structure" and verify how narrator is reading the content

Actual Result:

In narrator scan mode when user navigates to "example folder structure" using up/down arrow keys narrator is just reading as "Label1", "Image1" & "Image2" etc. but not reading the levels

Expected Result:

In narrator scan mode when user navigates to "example folder structure" using up/down arrow keys it would be better if narrator reads the levels along with reading the content

User Impact:

It will more useful and screen reader users will understand more clearly the structure if narrator reads the levels

Attachment for Reference

1154_ML .NET__Narrator not reading the levels for format.pptx

beccamc commented 3 years ago

@briacht and @romalpani Any thoughts here on what the language should be? The problem is that when going through the Images example it's hard to tell what's happening via screen reader. Saisree suggested Level 1, Level 2, etc. "Images, Level 1" "Label1, Level 2". Any thoughts?

image

romalpani commented 3 years ago

@beccamc This is an interesting one. Could we do what the narrator normally does for a folder structure? I used the 'Narrator app' in Windows to narrate a similar folder structure. I made some tweaks to what I heard, but what do you think about something like this?

"Tree View ... Images ... expanded Label 1 ... One of three ... expanded Image 1 ... One of two Image 2 ... Two of two Label 2 ... Two of three Label 3 ... Three of three"

beccamc commented 3 years ago

@romalpani That sounds good! How does the user know they're back up a level on "Label 2"? Is it because Image 2 is said as 2/2?

romalpani commented 3 years ago

Yes, I think that the count helps in giving the user an idea of what level they are at, in addition to when it reads out the "Label2 ... Two of three".

I have not tested this, but I was solely basing it on the way it reads out loud for normal folders in Windows.

beccamc commented 3 years ago

image

SaisreeDandamudi commented 3 years ago

@beccamc Verified the bug in the latest build and the issue is partially fixed. Narrator is reading Label1 1 of 3 expanded. Now it is understandable for screen reader users but I think that it should not be announced expanded/collapsed Reading Expand/collapse may give that idea to the user that these are actionable content. So reading expanded is not correct in this case as it is just a non-actionable text.

Since this is Sev4 bug. As per C & AI Bug Severity process Sev4 bugs does not effect your SLA. So you can add them to your backlog

beccamc commented 3 years ago

Rohan is setting up a meeting with Dante on what the experience here should be.

beccamc commented 3 years ago

Current proposal:

Sample Folder (Sample Folder, Level 1) Sub Folder Label 1 (Sub Folder 1, Level 2) Image 1 (Image file 1, Level 3) Image 2 (Image file 2, Level 3) Sub Folder Label 2 (Sub Folder 2, Level 2) Sub Folder Label 3 (Sub Folder 3, Level 2)

beccamc commented 3 years ago

Here is the build for testing https://devdiv.visualstudio.com/_apis/resources/Containers/7859052/VSSetup.Dev16Insertion?itemPath=VSSetup.Dev16Insertion%2FMicrosoft.ML.ModelBuilder.vsix

beccamc commented 3 years ago

vzhuqin, you don't need to verify this one. @SaisreeDandamudi from the accessibility team will need to verify and close this. Thanks!

beccamc commented 3 years ago

Please use public build https://marketplace.visualstudio.com/items?itemName=MLNET.07

beccamc commented 3 years ago

@SaisreeDandamudi can you please verify this? Thanks!

beccamc commented 3 years ago

@NiharikaTipirishetty It seems like our old accessibility contact isn't here anymore. Could you validate this as well? You can check it on rel/candidate build 16.7.2.2141903.

NiharikaTipirishetty commented 3 years ago

Issue is fixed. In scan mode, Narrator is clearly announcing the levels of folders in example folder structure format. Hence closing this bug.

1154_Narrator_fixed