RustAudio / baseview

low-level window system interface for audio plugin UIs
Apache License 2.0
283 stars 58 forks source link

add screen reader support via AccessKit #200

Open Timtam opened 1 week ago

Timtam commented 1 week ago

Hello,

I recently tried to implement a VST plugin using nih-plug (https://github.com/robbert-vdh/nih-plug). nih-plug uses baseview in combination with one of three different UI frameworks, Iced, Vizia and Egui, of which Vizia and Egui already support accessibility when run via winit, but not via baseview. In order to provide screen reader accessibility under Windows and Mac (and presumably other platforms too), the AccessKit crate is used. I opened an issue over on the nih-plug issue tracker (https://github.com/robbert-vdh/nih-plug/issues/174), and @DataTriny stepped up with some really helpful insights.

On Windows, accessibility should be activated when a window receives the WM_GETOBJECT message. As you can see in baseview/src/win/window.rs in the function wnd_proc (starting at line 121), this message is simply ignored. To directly integrate AccessKit, one must call accesskit_windows::Adapter::handle_wm_getobject whenever this message arrives.

These are only instructions for Windows, I assume corresponding measures need to be taken in order to support Mac OS accessibility as well.

It'd be great if someone could take a look at this, I myself am way to unexperienced with that I think, although I'd be happy to help wherever I can. Testing can be done natively under Windows via Narrator and VoiceOver under Mac OS. Alternatively NVDA, a free and really popular screen reader, can be used under Windows too.

Please let me know if you need more information.

Thank you.