search

surge-synthesizer / surge

Synthesizer plug-in (previously released as Vember Audio Surge)
https://surge-synthesizer.github.io/
GNU General Public License v3.0
3.17k stars 400 forks source link

Use JUCE Accessibility branch to bring reasonable accessibility to Surge XT 1.0 #4616

Closed baconpaul closed 2 years ago

baconpaul commented 3 years ago

this is a checklist @baconpaul is keeping he hopes summarizing his reminders of what to do based on this thread. (Check marks mean things he has done on a branch but not merged.)

https://forum.juce.com/t/juce-accessibility-on-develop/45142/83

ScottChesworth commented 3 years ago

Great to see this assigned to a milestone. I'll subscribe to the thread, tag other folk with helpful knowledge once there's something to test/code to examine, and am very much looking forward to making filthy noise in the name of progress. :)

baconpaul commented 3 years ago

wahey

A start

baconpaul commented 3 years ago

but i'm gonna need a lot of help from people who know how things 'should' work.

ScottChesworth commented 3 years ago

but i'm gonna need a lot of help from people who know how things 'should' work.

You've struck luck. I happen to be a natural-born griper about how things "should" work. :) Will take this for a spin first thing tomorrow and start jotting stuff down. Do you want all feedback collated in this here thread?

pitermach commented 3 years ago

Hello,

Ever since I heard about the Juce accessibility changes and that you were porting Surge over I've been following this issue with interest and I'll be more than happy to provide feedback. ☺️

I'm not sure if this build is already supposed to have any accessibility or not. I tried installing the latest nightly snapshot (following the "ReleaseXT build status link in the readme), installed it on my Mac, but VoiceOver didn't see any controls either in the standalone app or while running inside Reaper. I'm not sure if this was the right build (the build status page didn't seem to provide any artifacts) or if the CI isn't building against the development Juce.

On 3 Aug 2021, at 18:01, Paul @.***> wrote:

but i'm gonna need a lot of help from people who know how things 'should' work.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/surge-synthesizer/surge/issues/4616#issuecomment-891968240, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABAABTKURPG6EZ7VBWK536TT3AHG5ANCNFSM45YHRRLA.

baconpaul commented 3 years ago

Hi!

So thanks both @pitermach and @ScottChesworth. And welcome to the team!

On workflow

  1. Yes please drop stuff here. If I end up fixing it I may edit your comment to strike it out or may move things into a checklist but reporting here is absolute best bet
  2. When you report please include the version you are running and the OS. You can find this in the standalone by looking at the title bar, or you can always do menu / about and then copy the information from the about screen (although this isn't accessible yet)
  3. I'm a noob at writing accessible C++ so help on tools, how you test, what you expect is super welcome, but being more explicit than less would be useful. For intstance, I don't know how to use voice over. I've been testing everything with macOS accessibility inspector though
  4. Before you test too much, wait for the next nightly which will give (in acc inspector at least) a basic set of actions to sliders and multi-switches in surge
  5. You can always grab the latest nightly release here: https://surge-synthesizer.github.io/nightly_XT
  6. Our discord is also an active space and you are all welcome in the surge development channel to chat if you want. but GitHub works too!

Finally, let me emphasize again that I really want surge to work well in this regard, but I don't know what "well" entails for a synth. So happy to collaborate with you in a patient and friendly fashion with the goal of getting this from "nothing" a month ago to "bad but shows something" with todays pushes to "pretty good actually" by our release in late oct or nov.

baconpaul commented 3 years ago

(moved)

baconpaul commented 3 years ago

Nightly with some more basic accessibility is in place now I might do a bit more this week greatly appreciate any feedback on this super-duper-alpha stage

baconpaul commented 3 years ago

Notes for self playing with that nightly on to do next (x means at least done in my branch)

ScottChesworth commented 3 years ago

Took a super quick dive into Surge XT - 0.9.nightly.62261f2 on Windows 64-bit, investigated VST3 and standalone with NVDA and Narrator screen readers. I'm intentionally testing two pieces of screen reader software here, because the JUCE recommendation of testing with Narrator is insane. Whilst Narrator is a decent screen reader nowadays, hardly anyone uses it in production environments, and I doubt people will learn a whole new screen reader just to use one instrument. Anyway, here's initial observations:

  1. There's no tab order to navigate the GUI yet. This will be a showstopper for most folk on Windows. Object hierarchy navigation is really only something that advanced screen reader nerds know how to do efficiently outside of Narrator (which uses hierarchical navigation by default).
  2. The GUI is HUGE! I was able to clatter around in the object hierarchy and tweak, but man, organizing this in an object-based navigation paradigm is gonna take some thought.
  3. You're exposing tons of DAW automation parameters already, and most of their labeling seems pretty solid at a first glance (will probably have some productivity tweaks to suggest as I start spending more time with them). This comprehensive approach to params is a very good thing, because most accessible DAWs present a productive way to navigate and in some cases search these from the keyboard. Thus, deep level tweaking is already possible to some extent, albeit divorced from the UX you're presenting in the actual GUI. It's worth noting though that a UX of tweaking via DAW params is something that screen reader users are already going to be accustomed to, irrespective of their chosen platform and DAW.

For what it's worth, I'd suggest improving keyboard access to the patch browser window to be a priority. Given that you're offering a huge selection of patches and they're already pretty tweakable via DAW params, I'd say this thing will be way easier to test the sooner screen reader users can get stuck into the patches and actually start using it in situ. Ergo:

  1. I wasn't able to find a way to open the patch browser with keystrokes. Had to use OCR and do some virtual mouse trickery to get focus into the window.
  2. Once I got focus landed, the browser read pretty well here. Categories were navigable with arrows, their states (collapsed/expanded) were reported properly.
  3. Here's a showstopper, as soon as I hit Enter on a patch to load it, keyboard focus got thrown out of the patch browser. DOH!

Apologies for length, figured it'd be a good idea to include some extra explanation in place for ya, at least to start with.

baconpaul commented 3 years ago

Super useful We have a control group heirsrcht on params which we may be able to project into accessibility. Let me see what I can do

agree on patch browser. The version you tried had zero accessibility there. The one after that has quite a bit more including patch and category jog buttons being acc but I need a show menu and reader on the actual patch display

do you happen to know how juce handles grouping snd tab order? If not I will dig around

ScottChesworth commented 3 years ago

I need a show menu and reader on the actual patch display

Oh, so is the patch browser a popup menu that closes after you've chosen a patch? I'm flying blind here (literally). That'd explain the loss of focus I guess. Is there any mileage in being able to load a patch and keep the browser open? Maybe something like RightArrow on a patch to load it and maintain focus, or Enter if you're sure that's the one you want and the menu closes? That way you're not yanking away an established UX, but you'd keep the button-bashing to a minimum.

do you happen to know how juce handles grouping snd tab order? If not I will dig around

Sorry, no. I don't have dev chops here beyond a bit of tweaking/breaking other people's output. Can link you to a JUCE-based app and plug-in where tab orders have been implemented though: https://github.com/sonosaurus/sonobus

mkruselj commented 3 years ago

So the current patch browser is just a popup menu, but we're also working on tag-based searchable patch browser/database, that will most certainly be able to be kept open after loading a patch.

ScottChesworth commented 3 years ago

Ooh, liking the sound of that. Think I recognize you from the Reaper forum btw, unless there are two evil dragons... :)

mkruselj commented 3 years ago

In the best Highlander fashion - "there can be only one!" 😀

pitermach commented 3 years ago

Gave Surge XT - 0.9.nightly.62261f2 a shot on an M1 MacBook Air with VoiceOver.

Firstly, +1 to the idea of grouping controls together. THis I would say is even more important on Mac, where VoiceOver navigates through controls in a hierarchy, so when you move onto a group you just get told what it is and you can either move inside to see its controls or skip over it.

The second major issue I noticed is that the various controls need to be labeled correctly. Right now, VoiceOver was reading every radio button's label as just "select", while every slider was labeled as "adjust", without telling you what it is. The labeling of sliders goes in hand with providing real world values to the screen readers but you already have that on your list so that's great. ☺️

Bringing up menus worked fine, for the most part. In the standalone Surge app, the menu opened up and gained focus automatically and I was able to navigate it, however when Surge was hosted inside Reaper the menu would appear, but not get focused requiring me to manually find it with VoiceOver's "Window chooser" feature. I don't know if this is a Juce or Surge bug as this is the first Juce based plugin I seriously used on Mac, or whether the same thing also happens on Windows so I'll need to do more research there. On the topic of menus though one thing you'll almost certainly want to do for the Windows side is allow pressing either Shift+F10 or the "Applications" key to bring up the menu for the focused control. This is because on Mac this is handled by the screen reader, which has a shortcut to bring up the menu for a control, while on Windows screen readers rely more on the keyboard shortcuts provided by the OS for most actions, Shift+F10 being one of them.

Regarding modulation, as this is something not really doable through automation parameters so I think people will want to use the UI often, I'm guessing the "orange arrow" you're referring to indicates that a parameter is being modulated by something? If that's the case, then I think the simplest thing to do would just be to inform people about it by changing the accessibility label for the control. So for instance for the very first slider I see which is for the global volume, if it's being modulated by something its label could become "global volume modulated by X"

pitermach commented 3 years ago

Also, I figured I'd give a very quick explanation on how to use VoiceOver for testing, which I think might be helpful for you to see how the accessibility data you're giving it is being interpreted. ☺️

For completeness, testing on Windows with either Narrator (what Juce used) or NVDA (what most people use), is slightly different in that Windows users in contrast rely on the OS's keyboard navigation, (Tabbing through the controls, arrow keys to make adjustments, spacebar to press things), while the screen reader is usually just along for the ride speaking whatever got focused. Windows screen readers also have keys to step through the objects on screen, (For Narrator it's Capslock-Right and Left, with enter to activate something), but these usually are used when an app doesn't have very good tabbing support.

mkruselj commented 3 years ago

I'm guessing the "orange arrow" you're referring to indicates that a parameter is being modulated by something?

Orange arrow is on LFO modbuttons, they focus the bottommost panel in Surge to show a particular LFO's controls.

baconpaul commented 3 years ago

OK so lots of great information here. Thank you and @ScottChesworth and @pitermach officially "welcome to the surge synth team"!

So here's some responses

  1. All - Clearly hierarchy and tab order are critically important. We have internal data structures which let us do natural groupings of controls (the controlgroup types) and have a natural ordering in parameter space also, so we have the data in the synth but it may be a bit of work to figure out how to expose this via JUCE. For my edification, how would I know if I had hierarchy and tab order right with your tools?
  2. @ScottChesworth between the version you tested and the version after a lot more components 'lit up' as accessible, including the always-on patch and category jog buttons. But the patch browser itself still doesn't have an accessibility handler. I plan to add one in my next dev sprint (which will probably be tomorrow) and when I do I will add a ::showMenu handler (which will pop up the current menu @mkruselj ) and a ::press handler (which will open the new patch browser).
  3. All - the "orange arrow" is a UI component on the modulation buttons. In the 62261f version the modulation buttons at least explain their names but the mod button is a multi-function button which has an activity to arm it (click) but also an activity to select it as a target. If you have LFO 2 armed and you want to modulate LFO 1, you press the 'orange arrow' section of LFO1. I need to think about how to have a single component have multiple accessibility actions though.

Right now, VoiceOver was reading every radio button's label as just "select", while every slider was labeled as "adjust", without telling you what it is.

  1. @pitermach this fact bummed me out a bit. I followed the JUCE protocol that the description is a single word and the title is the name of the control. So if I select one of the sliders in the accessibility inspector I see

Label: Adjust Title: A Filter 2 Cutoff Value: 0.63854 Type: slider

and it seems voice reader is just giving you "Adjust".

For those controls I can make the label the same as the title, or make the label "Adjust A Filter 2 Cutoff" (or "Adjust Scene A Filter 2 Cutoff" even better). but it seems this JUCE documentation for getDescription is wrong:

This may be read out by the system. It should not include the type of the UI element and should ideally be a single word, for example "Open" for a button that opens a window.

  1. Finally I'm going to take my checklist halfway down the thread, add it to the top of this thread and keep editing it with the outstanding work here. This is excellent feedback and a guide, and appreciate you trying the super-alpha of this. I'm sure we can make something good and am very happy for the testing and design work you are doing!
baconpaul commented 3 years ago

Oh and finally, I don't have a dev day today so doubt there will be a new nightly today (or at least not until late tonight). I'll post with changes when I have stuff up and will read any responses too. Thanks!

pitermach commented 3 years ago

Thanks for the warm welcome! ☺️ I managed to figure out a few more things and have more ideas as a result:

Re: The titles, it turns out that they are kind of visible, but only if you "interact" with a control (I explained what that is a few comments back), but while navigating around the interface it's the labels, states and values that get spoken. Interestingly enough, on Windows the situation is different, and there the "title" attribute works how you expected it to which is probably why @ScottChesworth didn't mention any missing label problems.

Because I was able to figure out what all the controls are I started to experiment with them and noticed some things with the ones reported as "radio buttons".

What I noticed right now is that, pressing on something reported as a radio button usually didn't do much, but a right click often presented a lot of options, but not always - the scene selection is a good example. So, I had a brief look at the manual and I'm guessing both a selection of the scene as well as the play mode are a row of buttons, only one of which can be active. This doesn't seem to be exposed this way for accessibility right now, but it's definitely a good use of a radio button role. Here's some ideas on when to use different roles:

Regarding modulation, what might be simplest to do in this case is having the main role of the modulator be the most common action, like having it be armed or not, and then providing access to setting its target through a hotkey. I think this might be the first instance of a synthesizer plugin that provides an accessible way to use a more advanced modulation so you're truly breaking new ground here ☺️

What I mean by this would be something like this: You tab to or get to the modulator you want to use, like LFO 2, and then press some key to set it as a source, which could be confirmed with an announcement of some kind. Then you'd move to the parameter you want to modulate, like LFO 1, and press the same key again which would select it as a target. This would match the workflow of someone using a mouse closely enough I think and then subsequent changes to things like the modulation depth could be done from the right click menu from what I gathered by looking around menus and reading the manual. I also noticed you can quickly set up modulations by right clicking on most sliders as well so I suspect this is how most people will end up using it.

baconpaul commented 3 years ago

Right so on the selectors yes they are a peculiar widget. They are a single compound widget with multiple discrete states. Think of them as a slider over integers. The present in the UI visually as a stack (sometimes) but also the filter selector looks like a set of small buttons across the bottom with a picture of the routing diagram above. I picked 'radio button' since that's what they are most like, but indeed i didn't expose the children as components. I think I can mock up fake accessible sub-components of the widget though to make it more radio like. Let me dig in the API a bit and add it to my running list above.

The mac-vs-windows thing is a bit irritating but easy to work around. I think what I'll do for now is make a single entry point to set up accessibility on a widget with a parameter and then use a #if mac / # if windows approach to set the description differently to include title or not, and also comment over on the juce forum that we ran into this problem. Interestingly in the mac accessibility inspector I see the title in the top but in the hierarchy I also just see a row of 'adjust' so I now know how to quickly check if this works (and will leave it unchanged on windows).

On the modulation component. The thing is the button is really two components. An "Arm" component and a "select if someone else is armed otherwise be disabled" button. Again we compress that into a single widget but if I can figure out how to make fake accessible sub-components without adding juce components to the hierarchy that woudl be best, so then a modulator accessible button is a group named "LFO1" which contains two buttons, one named "ARM" and one named "SELECT FOR MODULATION" or something like that when the single component presents to accessibility.

I also found 45 minutes of dev time this morning and have just added an accessible interface to the patch selector which shows the name of the patch and allows you to launch the (classic) menu and the (new wildly alpha basically doesn't work yet) patch browser we are developing for XT. THis will address several of @ScottChesworth 's issues where he had to revert to OCR I think. New nightly in an hour or two.

pitermach commented 3 years ago

Great. The idea of having multiple widgets probably the most sense to me as well since they're 2 separate controls in a sense

baconpaul commented 3 years ago

Great. The idea of having multiple widgets probably the most sense to me as well since they're 2 separate controls in a sense

Yeah but for skin-engine reasons I want them as one juce::Component so gotta figure out that artificial-sub-component API and if it is exposed (I know it is in NSAccessibility).

But worst case I can just make the components if i have to.

baconpaul commented 3 years ago

OK so should be a new nightly in about an hour (821c2a8) which fixes some of the rudimentary issue you raised.

I have a start on a branch which is doing way better grouping as well. There's actually an easy way to do it it turns out. At first this will give us a grouped hierarchy but I can go from there to focus order (though focus order will come second).

Welcome any feedback on any nightlies which come but at least wait until 821c2a8 to do a test since it fixes the voiceover names on mac as well as the patch browser stuff. Or may want to wait until weekend when I hope to have rudimentary grouping in place.

baconpaul commented 3 years ago

Just merged a change which improves traversal order. When the 0fcdf44 nightly is ready (or any after that) would be curious is I got it right :)

pitermach commented 3 years ago

Thanks for working on all of this.

I had a look, and I mostly think the groups are good. I would group the control to select the oscillator in the "oscillator controls" group though, and ditto for the waveform button because right now they're very far away from the controls they relate to. So it takes a lot of keystrokes to select the oscillator you want to edit and then return to its controls.

Also, I'm happy to say the labels work nicely with VO now. One thing I would tweak slightly is remove the "adjust" and "select" words from the labels, because they're a bit redundant with speech. Hearing the name of a control as well as its type is enough for someone to figure out what it does - For example, VoiceOver might say something like "global volume slider", which already tells you that it can be adjusted, rather than having to hear "adjust global volume slider" which is how it sounds like now. So having to wait for the speech to get past the "adjust" and "select" bit many times as you search for the right control adds up.

baconpaul commented 3 years ago

Oh that's a super good point yeah. The automated grouping makes that a global control (it selects 'which oscillator' to map params to on screen) so it doesn't group with the oscillator controls programatically. But it s a one line fix, as is the suppression of the word 'adjust' on macOS. I'll add both to my list.

Will ping this thread when there's a new one worth testing. Finishing a couple of the controls and stuff is my next step.

Appreciate the deep testing and thoughtful feedback!

baconpaul commented 3 years ago

Just a quick ping on this thread. I just merged a change which has the hierarchy and I think labels all correct. Some of the controls still don't respond to value changes or format value correctly, and the modulation buttons and radio buttons don't have sub components, but the bulk navigation and grouping is I think roughly done.

May have time to address some more of the access issues this weekend. Checklist at top is kept up to date.

pitermach commented 3 years ago

Yep, the grouping's looking good. ☺️

baconpaul commented 3 years ago

Wonderful. Hacking away on the other controls this weekend.

baconpaul commented 3 years ago

Anyway did a few more pushes. Taking a couple of days away but it seems most things except modulation can at least be selected and edited in the inspector now and are all in natural units. I also changed the multi switch to a radio group. Probably 7 days or so before I get back to accessibility but any feedback in interim always welcome and the help so far has been absolutely critical. Thank you!

pitermach commented 3 years ago

No worries, take your time.

Here's some more feedback then. I grabbed 2 builds, nightly.049dd11 yesterday when I happened to see some commits, and nightly.78be91c an hour ago when I came back from work and saw your message. This ended up being a really good thing, because on both Mac and Windows in the first build the radio button controls displayed correctly, while in the latest build on both platforms while you can see the group, none of the radio buttons that should be inside show up so you can't adjust anything, so something must have caused a regression.

In the builds where the radio buttons worked, there were two things I noticed which will still need a look at. One is that some weren't labeled right, IE the buttons to select a scene were all read out as "non-param tag", ditto for the patch/category jog controls and probably a few others I'm forgetting. Those previous/next patch controls actually should probably be changed to regular buttons, since they perform a function but don't stay pressed in after. Lastly with the radio buttons, none of them indicate that they're "selected/checked", so you can't really tell what option is selected from the group. This is also true about the toggle switch controls, which always appear as "off" or "unchecked" to screen readers.

Re: Sliders, 90% of them seem to work well and read their actual value. On Windows, NVDA still seems to ignore the real value and use a different scale, but as this doesn't happen with the other windows readers I tried (Narrator, JAWS, ZDSR) I suspect this is an NVDA issue especially because it also happens in at least one other Juce plugin (Pianoteq). CC @leonardder, you're familiar with UIA and you followed the juce developments, should we take this up with NVAccess or the juce people?

More about the sliders, there were a couple that behaved weirdly when I adjusted them, mainly in the effects section. The Effect slot slider would adjust, but every time I did, both on Mac and windows after a single adjustment something would cause the focus to be lost and the screen reader would return to the very first control in the window. The other issue was with some effects. As a test I put a “reverb 1” in one of the slots, and most sliders would adjust OK, except the reverb type which wouldn’t increase or decrease, although I was able to use the menu to select a different option. Most other effects didn’t have this issue - I tried things like resonator, neuron, Reverb 2, both EQ’s and some AirWindows effects, and all sliders in those worked right.

Finally, on the topic of the sliders, something else that’ll need to be done is allowing them to be adjusted with the arrow keys when they’re focused, same thing for the radio button groups. This is again mainly because of the Windows/Mac differences that I mentioned before, but it might also let you solve some of the issues you were thinking about - IE, pressing space or enter on a slider could reset its value, which would let you not worry about making an accessibility action for that, while using the arrow keys with a modifier like shift or alt could adjust the secondary modulation amount slider if the parameter is being modulated.

LeonarddeR commented 3 years ago

Re: Sliders, 90% of them seem to work well and read their actual value. On Windows, NVDA still seems to ignore the real value and use a different scale, but as this doesn't happen with the other windows readers I tried (Narrator, JAWS, ZDSR) I suspect this is an NVDA issue especially because it also happens in at least one other Juce plugin (Pianoteq). CC @leonardder, you're familiar with UIA and you followed the juce developments, should we take this up with NVAccess or the juce people?

It looks like the sliders implement both the ValuePattern and RangeValuePattern. The RangeValuePattern exposes a numerical value whereas the ValuePattern exposes the real value. From what I can see, it looks like code in NVDA needs to be swapped in order for this to work correctly. I have to take a look at whether this is an authoring error in Juce or NVDA.

baconpaul commented 3 years ago

Thank you folks. Like I said few days break but will definitely look at all of these next 7 days. Radio buttons may be quick too so will peek there first.

baconpaul commented 3 years ago

OK I found 20 minutes this AM to restore the MultiSwitch radio buttons (d'oh! Sorry bout that) but also to make the radio button elements and the switch elements support the checkable/checked status just like the juce radio buttons. This makes the subcomponents have a value which matches their checked state. Should be in a nightly in the next 90 minutes or so unless CI fails.

The FX thing you report is really hard. Surge has some conditions where it rebuilds the entire UI. This is partly a remnant of the VSTGUI code base. I will struggle with restoring focus in that case. I'll ponder though.

Finally on arrow key for sliders. Are these just regular keyboard events? We have resisted supporting keyboard events in the plugin since they conficlit with bindings in the DAWs so often. Or do accessibility clients send different messages? I see when I use mac accessibility inspector my sliders all have jog-by-range-increment buttons added.

Thanks for all the feedback as always

pitermach commented 3 years ago

Can confirm the button states work and the groups are once again visible, thanks for the unscheduled fix ☺️

Regarding your question with sliders, I'm not familiar with how Juce does this internally but I guess that's how it might work. In my experience, the keys you'd care about the most - tab, arrow keys, enter or space, should get past by the DAW - at least they are by Reaper which is what most people use on Windows, which also has an option in the menus to allow all other keys if needed.

Why this is so important once again goes back to the Mac vs Windows differences that I touched on before - Mac users mainly use VoiceOver specific commands to navigate and manipulate applications, (those are probably also the increment and decrement buttons you're seeing in the inspector which it internally presses). In contrast, Windows screen readers let the OS and applications handle the keyboard support for navigating and manipulating the UI. Technically, Windows screen readers also have commands to navigate the accessibility tree like VoiceOver, but they're considered advance concepts and don't expose all of the functionality - afaik most just have a "perform primary action" command which is fine to press buttons or toggle switches, but on a slider it might only let you adjust it in one direction.

baconpaul commented 3 years ago

Very useful What I will probably do is have keyboard actions default off but activate if you ask for accessible api. Then have a user setting which lets you pick the “never / on accessibility / always” option

we have had a squillion problems with key presses and the wide variety of daws and platforms but above strategy would invisibly work for you and not break non-acc users in idiosyncratic daws

pitermach commented 3 years ago

THat sounds reasonable. So by your message I guess you can check when a screen reader is running and set this option automatically? If so then I think it'll work great.

On 10 Aug 2021, at 19:01, Paul @.***> wrote:

Very useful What I will probably do is have keyboard actions default off but activate if you ask for accessible api. Then have a user setting which lets you pick the “never / on accessibility / always” option

we have had a squillion problems with key presses and the wide variety of daws and platforms but above strategy would invisibly work for you and not break non-acc users in idiosyncratic daws

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/surge-synthesizer/surge/issues/4616#issuecomment-896154475, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABAABTOKTZ26V3F4VFQYJGDT4FLP3ANCNFSM45YHRRLA.

baconpaul commented 3 years ago

Yeah exactly. There’s an api when an accessible client queries the ui

ScottChesworth commented 3 years ago

FWIW Paul, in general, it's kinda frowned upon to serve up an alternative UX based on detection that a screen reader is running. The objection is most often raised by a noisy minority of screen reader users who are sensitive to being treated differently, and/or are conscious of that additional data point being collected and associated with their identity. In this case, your strategy makes perfect sense given the amount of platforms and hosts you're supporting and I'd say it's the right move to make. However, if you work on other software outside of Surge (particularly if you work on websites where data collection is involved), the same move would likely tickle the angry blind mob in all the wrong places. Just figured I'd throw a note in seeing as this is your first foray into accessibility. Sometimes there's a weird balance to be struck between what makes sense in terms of implementation and what reception its likely to receive, because when a product first becomes accessible, word of mouth is basically everything. I know, weird world.

In other news, I'm horribly behind on nightlies. Will catch up and add feedback on current progress asap.

baconpaul commented 3 years ago

Yeah I get that. Surge collects zero data but I’ve worked on web projects where full always on accessibility is the norm and a hard requirement (but I didn’t code or test those)

Appreciate the tip. Here it really is that the arrow key capture can screw up daws so we need to be careful. I wish it weren’t so.

baconpaul commented 3 years ago

Oh also I am going to have to make some fake elements for accessibility only since the accessibility api is “smaller” than the full screen api. For instance a button which acts differently based on which part of the button you press needs to present as 2 buttons to accessibility. Things like that seem pretty standard though right?

ScottChesworth commented 3 years ago

In UIA, there's a SplitButton control you could use for that: https://docs.microsoft.com/en-us/windows/win32/winauto/uiauto-supportsplitbuttoncontroltype No clue if/how that translates over to accessibility APIs for other platforms though.

baconpaul commented 3 years ago

Yeah I don’t think that comes through in the juce api. I was just gonna make the modulators a group with a select arm and modulate-onto sub button

ScottChesworth commented 3 years ago

That sounds like it'll be intuitive yep.

baconpaul commented 3 years ago

Just a ping - I haven't forgotten this thread but once I got a 'proof of concept' working in our alpha (thanks to all your help) I went on to the other high-risk-to-emerge-from-alpha issues in our book of work for a bit. will return to this soon though and will tag here when I do.

baconpaul commented 3 years ago

Hi folks

Over the last month or so we've been chipping away on the finishing XT touches towards a beta. In the last few weeks I re-worked quite a lot of the accessibility layer and also started testing it by scripting mac accessibility on with python.

Right now, other than setting modulations, I think the core features of the synth are all exposed in the screen reader. I may get modulations worked out shortly also. We are planning a long beta period across Dec and early january where folks test, but in the event that you have some time, especially with the thanksgiving week coming up in the US, wanted to just flag this issue and say it may be a good time to test a nightly again.

The nightly is here: https://surge-synthesizer.github.io/nightly_XT

that page says "alpha" but it will say "beta" very very soon.

Thanks as always for helping with this effort

pitermach commented 3 years ago

Hi Paul,

Once again a huge thanks for working on this. testing I had a bit of time to do some more testing on both platforms over the last few days, and one of my friends also started using it in the wild so there are definitely people enjoying it. ☺️

I don't really think I have any complaints with the main controls and how they're laid out. Once you get used to the groups you can get to what you need relatively quickly especially if you know your screen reader. I was also able to set up some basic modulation using the right-click menus on the sliders, so even that can be done to an extent even now.

Apart from refining the modulation accessibility, the other 2 points I can think of that still need improvement is adding the tab order for controls and making the modern preset browser more accessible.

I already explained the first point with tabbing a few posts back, but what I noticed even now is that some controls can in fact be tabbed to. Specifically, in the main interface if you tab the focus moves to the modulation overview button and back to what screen readers are calling the "Main Frame". Also, if you open the new preset browser or the modulation overview dialogs, these two can be tabbed through, and the lists in the preset browser can be navigated with the arrow keys. I'm not sure what makes these controls different, but perhaps you can implement whatever they're doing to make the other UI elements reachable from the keyboard this way.

Regarding the preset browser, there is already some accessibility groundwork. On both Windows and Mac screen readers announce the search box as well as the category tree and preset list, and if you move through them with the arrow keys at least on Mac the row numbers, as well as the collapsed/expanded state for the tree get spoken. However, screen readers don't see any of the other information like the preset or category names. This issue also seems to effect the lists of inputs and outputs in the settings dialog of the standalone application.

The other thing is that there isn't currently an easy way to load a preset. I think it'd be a good idea if pressing enter on a patch would load it since the list can already be navigated from the keyboard anyway, this would probably be useful not just to blind folks. But also it can't be activated using the screen readers activate command. On Mac, that section of the program is for some reason completely unreachable with the VoiceOver cursor and the only way I was able to read it was by tabbing through the controls, while on Windows the only way I was able to load a preset through the browser was using a simulated double-click

Sorry for the wall of text! ☺️

baconpaul commented 3 years ago

This is great feedback and I will update my todo list here in the beta period. yes the patch tool has no values now and some of the sub screens are also not accessible yet.

to load a patch I have been using the show menu action on the patch selector. That brings up the menu which at least on mac I can traverse with a screen reader. Does that not work in your environment?