slint-ui / slint

Slint is a declarative GUI toolkit to build native user interfaces for Rust, C++, or JavaScript apps.
https://slint.dev
Other
16.94k stars 568 forks source link

Accessibility: Add support for performing actions on elements #2872

Closed tronical closed 3 months ago

tronical commented 1 year ago

With accessibility APIs there are ways that assistive tech like screen readers can activate elements "programmatically". This is sometimes emulated by the system using a synthetic mouse click (VoiceOver on macOS seems to like doing that), but for example the AccessKit APIs have dedicated actions that can be run, such as "Increment", "Decrement", "Expand", "Collapse", "ShowContextMenu", "SetValue" (slightly more generic), or even "Default" (presumably press on a Button, for example).

We should allow defining such actions in .slint and invoke them from the run-time library if the accessibility APIs request for their invocation.

Perhaps we can use implicit callbacks for this?

accessibility-action-expand => { my code here to expand }
accessibility-action-increment => { self.increment(); }

If such a callback is defined on an element, then we can report to the OS that this action is supported and invoke correspondingly.

ogoffart commented 1 year ago

Another possibility:

accessibility-action(action, value /*: string*/) => {
   if (action == AccessibilityAction.expand) {
       // ... 
       return accept;
   } else if (action == AccessibilityAction.increment) {
      // increment
      return accept;
   } else if (action == AccessibilityAction.set-value) {
      root.value = value;
      return accept;
   } else { 
       return reject
   }
}

But this seems more verbose than many accessibility-action-* callbacks and also better types. (and in the runtime, the compiler can anyway lower both solution to the same code, so runtime consideration doesn't matter here, it's mostly what would give the better API)

ogoffart commented 7 months ago

I had some thought about it, here is what i think we need to do:

Runtime:

Compiler:

Widgets:

ogoffart commented 3 months ago

This was done in 1.6