ComposePanel does not override `getAccessibleContext`

[rpaquay]

Describe the bug ComposePanel inherits from JLayerePane, but does not override getAccessibleContext, meaning it does not expose any child component and, in particular, its _composeContainer to the Java Access Bridge API. This means screen readers cannot navigate the hierarchy of Compose components contained in it.

The fix is probably to override getAccessbleContext in ComposePanel with the following methods

  override fun getAccessibleContext(): AccessibleContext {
    if (this.accessibleContext == null) {
      this.accessibleContext = AccessibleComposePanel()
    }
    return this.accessibleContext
  }

  private inner class AccessibleComposePanel : AccessibleJLayeredPane()  {
    override fun getAccessibleChildrenCount(): Int {
      return if (_composeContainer != null) 1 else 0 
    }

    override fun getAccessibleChild(i: Int): Accessible {
      return _composeContainer?.accessible ?: throw IllegalArgumentException()
    }
  }

Affected platforms

• Desktop: Windows & macOS

[m-sasha]

Can you provide a complete example of the problem? This works correctly for me:

fun main() = SwingUtilities.invokeLater {
    val frame = JFrame()
    frame.contentPane.add(ComposePanel().also {
        it.setContent {
            Column {
                Text("Hello World")
                Button(onClick = { println("Click") }) {
                    Text("Click me!")
                }
            }
        }
    })
    frame.size = Dimension(400, 500)
    frame.isVisible = true
}

The accessible is not exposed via ComposePanel, but via skiko's HardwareLayer:

internal open class HardwareLayer(
    externalAccessibleFactory: ((Component) -> Accessible)? = null
) : Canvas() {

    private val _externalAccessible = externalAccessibleFactory?.invoke(this)

    override fun getAccessibleContext(): AccessibleContext {
        val res = (_focusedAccessible ?: _externalAccessible)?.accessibleContext
        return res ?: super.getAccessibleContext()
    }

}

[rpaquay]

I don't quite understand what HardwareLayer is about, but what I am trying to say is that I believe ComposePanel should behave the same way as a JPanel, i.e. it exposes the components it contains as Accessible through its implementation of getAccessbleContext. Here is the JPanel implementation:

    public AccessibleContext getAccessibleContext() {
        if (accessibleContext == null) {
            accessibleContext = new AccessibleJPanel();
        }
        return accessibleContext;
    }

    protected class AccessibleJPanel extends AccessibleJComponent {
        protected AccessibleJPanel() {}
        public AccessibleRole getAccessibleRole() {
            return AccessibleRole.PANEL;
        }

        // getAccessibleChildCount and getAccessibleChild are inherited from JComponent, and the
        // inherited implementation looks at the list of child component:
        int getAccessibleChildrenCount() { // Inherited from Container.java
            synchronized (getTreeLock()) {
                int count = 0;
                Component[] children = this.getComponents();
                for (int i = 0; i < children.length; i++) {
                    if (children[i] instanceof Accessible) {
                        count++;
                    }
                }
                return count;
            }
        }
    }

The reason it is important to link parent and children is that it is an expectation of the Java Access Bridge API (and screen readers using the API) that there is a top down Accessible tree that can be used to traverse all UI elements from the top level window (JFrame) to any element displayed in that frame. With the current implementation of ComposePanel, the Accessible tree stops at ComposePanel, and the container does not have the ability (afaik) to expose the sub-tree of Compose components it contains.

The use case we are looking is an IJ plugin (Android Studio) embedding a ComposePanel in a IJ platform ToolWindow. With the current implementation of ComposePanel, the Accessible tree is incomplete, i.e. the traversal "stops" at the ToolWindow container, and nothing is exposed via ComposePanel.

[m-sasha]

Can you provide a small reproducer app for this?

[igordmn]

@m-sasha, this is a bug from IntelliJ/Android Studio integration. It probably uses System.setProperty("compose.layers.type", "COMPONENT"). Could you try it with it?