grokys
commented
5 months ago I did start on an implementation of accessibility support for Linux using AT-SPI but I quickly hit a few issues:
- As mentioned elsewhere, there are no docs. When I tried even the XML definitions were in a non-working state. Looking at them now, they seem to be in a better state.
- I asked if there's any documentation and was basically told "the documentation is the GTK source code". I notice that there are now documentation comments in the XML definitions at least
- I had difficulty following and debugging what the GTK source code was doing
- The way that AP-SPI works with DBus seems to be rather unusual and at the time https://github.com/tmds/Tmds.DBus didn't seem to be able to handle it, which is where I gave up
It's unlikely that I'll personally have any time soon to resume work on this (unless a customer wishes to pay for the feature), but the aborted work I was doing can still be found on the ui-automation-atspi branch if anyone is interested in picking it up. Note that this code is probably really outdated and would need updating to the final implementation of the automation APIs in Avalonia.
webczat
kekekeks
maxkatz6
introduction
As a blind user which daily uses linux, I am unable to use any avalonia based applications because they are inaccessible to screenreaders, while such support exists for windows and mac. I would like accessibility support for linux to be added.
Some details
I don't know how avalonia architecture looks like, so can't comment on that part, however I can share some things I gathered about the linux accessibility stack.
Because avalonia does not use any ui toolkit under the hoot and is based directly on X (and possibly wayland, doesn't really matter that much), there are two options to implement accessibility:
Option 1: use ATK
Basically, that requires adding ATK as a dependency. In this case, for each avalonia control, you need a peer object implementing accessibility. This peer object is a GObject-based object inheriting from AtkObject and implementing one or more atk interfaces as appropriate for the object. The avalonia control itself doesn't need to be a GObject object, but one instance of a control needs to be bound to one instance of the AtkObject subclass appropriate for it. This probably can be done dynamically based on avalonia's way to indicate accessibility support for custom controls, if any.
Properties of a control need to be exposed via the atk object as appropriate, and also, any change in properties, state changes or events happening which influence the accessibility state need to be propagated through the atk peer.
ATK itself is not enough because the objects and events need to be exposed to at-spi. So, the library needs to load the at-spi2-atk bridge and initialize it first. I'm just not sure how do you integrate it with something that doesn't use the glib main loop, but you should be able to do it. I haven't done such implementation myself, I've just implemented a fake custom gtk widget with fake accessibility tree once, so my hands-on experience with that is limited.
Option 2: using at-spi directly
So, basically, we have a special accessibility bus, on which applications are exposed, and there is at-spi registry which knows about these applications. There is a library called libatspi, but it's intended for use by other side of the equation, like screenreaders and such like to inquire applications, and it can't be used to expose them. It's done directly over dbus.
Avalonia's job would be to find the accessibility bus (instructions: https://gitlab.gnome.org/GNOME/at-spi2-core/-/blob/main/bus/README.md?ref_type=heads), to connect to it and to expose it's accessibility tree as dbus objects. The last step seems to be registering an application's root object in the registry. There are probably few other special objects to be implemented like a cache...
Generally, dbus objects for accessibility tree need to implement the Accessible interface and one or more other interfaces as appropriate. The root application object also implements the Accessible interface along with Application interface, at least from what I can tell. Introspection data for all the interfaces along with some documentation can be found here https://gitlab.gnome.org/GNOME/at-spi2-core/-/tree/main/xml?ref_type=heads but at least in one place (the event.xml file) it's said that method signatures might be incorrect, so documentation might or might not be complete, and there is lots of info missing like what's the registry's bus name or object path that you need to use when registering an app. However, an app does not need a well known bus name.
According to documentation of accessible's GetParent method and based on local dbus introspection, the root application object should be called "/org/a11y/atspi/accessible/root" with an accessible role of application. All other objects are in it's subpaths, direct descendants seem to be windows (probably with a frame role).
There is also a special object /org/a11y/atspi/cache that is to be exposed by application and that should implement the cache interface, that seems to be for getting dump of the whole accessibility tree, unsure if it's implementation is required.
Registration of an application in registryis done using the socket interface (https://gitlab.gnome.org/GNOME/at-spi2-core/-/blob/main/xml/Socket.xml?ref_type=heads). One can probably detect registry disappearing and reappearing and re-register an app if needed. Unregistering is not necessary, closing dbus connection is sufficient. Socket is implemented on /org/a11y/atspi/accessible/root object of well known bus name org.a11y.atspi.Registry, or at least that's what introspection says.
I would happily provide more info but I have never tried to implement such a thing myself without ATK. But it could be an interesting exercise which I'd prefer to do without avalonia context if at all.
Testing
Testing of course requires playing with a screenreader like
orca, but there is also a tool showing accessibility tree calledaccerciserwhich can be used to debug things. It can also show accessibility events based on filters and do other fancy things which can be helpful when debugging, but note that some errors will only appear when using an actual screenreader as how it reacts to some events is not always that obvious.