Accessibility

[raphlinus]

The vision of druid includes accessibility, and I think it should drive architectural decisions. For example, imgui has a reputation for not being easy to integrate accessibility features; we want to avoid that.

I've looked a little into accessibility (most recently, watching the relevant SwiftUI video), but don't have a very strong sense what that would entail. One of the key features is integration with speech for visually impaired users, but there's also control over focus and selection.

There's a little bit of work in the Rust ecosystem, including jdm/accessible. It would be good to cooperate with others sharing the concern.

I've been thinking about this for a while, but the impetus to file this issue now was a yaah.dev blog post, and a link to resources in the Reddit discussion.

[futurepaul]

Want to keep this convo going. Some links from my travels: Apple uikit: UiKit Accessibility Element Apple appkit: NSAccessibility

Microsoft: Active Accessibility + UI Automation

Qt has an abundance of documented accessibility code... it's sort of bonkers! Qt accessible Qt ui automation (windows) Qt cocoa accessibility (mac)

Flutter uses a "semantics" class to annotate the widget tree

Overall the job seems to be:

1. create a tree based on the widget tree with appropriate metadata for accessibility (or, I guess, if possible, instrument the widget tree in such a way that it can double as this accessibility tree)
2. make that tree available to the platform-specific accessibility framework to traverse / call methods on

Anyway, I don't really know a lot about this subject, so if anyone has more resources / guidance please share!

[raphlinus]

I'll point out that creating an explicit tree is not quite a match for druid's current model; drawing is (with some differences) most similar to immediate mode. When we, for example, drop a widget, there's no explicit "unmounting," so there's currently no way to delete the corresponding node from the accessibility tree.

But this is something we can revisit. It sounds like further research into the requirements of accessibility is a good thing to inform these architectural decisions. Thanks for gathering the resources.

[ratmice]

I'm not sure about from a platform neutral perspective how feasible it is, but one thing which I would find very helpful if it could at least handle an event model where it were possible to perform menu item actions without chording modifier keys.

• Shift-down-up, could capitalize the next character input
• Ctrl-down-up c, and performing the copy operation.

I think most input systems of this sort probably don't build this into the application model, instead users who need this likely need to install custom input drivers, and perhaps feedback about the state of the input stack.

[richard-uk1]

I thought it would be useful to have a list of accessibility concerns. It's probably not the time to address them all, or it might not be that the UI framework is the best place to address them, but it's good to have them in mind. Here is my attempt at a list.

1. High contrast display
2. Keyboard navigation
3. Screen reader
4. "Plain english"
5. Consistent UI - e.g. help button is always in the same place.
6. ...

We would aim to address the needs of people who

1. are hearing impared
2. are visually impared
3. have cognitive difficulties
4. touch and motor difficulties
5. ...

Some of these would be well addressed in documentation, either written as part of druid docs, or links to external resources.

I don't have accessibility needs - it would be good if someone who does were to get involved with the project so they can advise what we can do - especially if there are low hanging fruit.

[giannissc]

This came out today and I think it might be useful: https://blog.gtk.org/2020/10/21/accessibility-in-gtk-4/