Accessibility: semantics(mergeDescendants = true) does not merge semantics of descendants

[ialokim]

From my understanding of Accessibility in Compose, the following code should

• treat the children in the UI tree as one element for accessibility purposes
• merge the semantics of the children (i.e., use the text value of all Text elements as Label/Value for a11y

fun main() = singleWindowApplication {
  Row(modifier = Modifier.semantics(mergeDescendants = true) {}) {
    Image(
      imageVector = Icons.Filled.AccountCircle,
      contentDescription = null // decorative
    )
    Column {
      Text("Romeo Montague")
      Text("April 02 • 1 min read")
    }
  }
}

The group is indeed recognized as one element, but the semantics (text values) are not merged.

[ialokim]

according to the documentation (emphasis added by me):

mergeDescendants: Boolean = false Whether the semantic information provided by the owning component and its descendants should be treated as one logical entity. Most commonly set on screen-reader-focusable items such as buttons or form fields. In the merged semantics tree, all descendant nodes (except those themselves marked mergeDescendants) will disappear from the tree, and their properties will get merged into the parent's configuration (using a merging algorithm that varies based on the type of property -- for example, text properties will get concatenated, separated by commas). In the unmerged semantics tree, the node is simply marked with [SemanticsConfiguration.isMergingSemanticsOfDescendants](https://developer.android.com/reference/kotlin/androidx/compose/ui/semantics/SemanticsConfiguration#isMergingSemanticsOfDescendants()).

[NupurM]

I tried out a couple of the talkback features in compose and found that some of them need newer versions of TalkBack apk. So you might need to try again with a new version.

Content Description Content Description using semantics Merge Descendants Click Label Role State Description Heading*

*Need newer version

[sebkur]

thanks @NupurM I think however @ialokim and I are not using compose on Android here but rather on desktop OS, particularly macOS in this case (as among the desktop operating systems the support for a11y seems to be furthest developed there). I think talkback is an Android app which we cannot use on the mac. There we've got the stock "VoiceOver" that is preinstalled on every mac as well as the Accessibility Inspector which comes with Xcode.

[EgorovDmitry]

@igordmn I've found that this issue affects not only desktop but also iOS application. .clickable modifier that works like mergeDescendants = true gets only the first available text from descendants, I have this structure and I have nothing in label in iOS accessibility hierarchy. If I set contentDescription to null it will help to passthrough text from descendants, but only the first text.

Row(
        modifier = Modifier
                .testTag(CalendarsTestTags.calendarItem)
        .clickable { onEntryClick(item) }
) {
    Icon(contentDescription = "")
    Spacer(modifier = Modifier.width(4.dp))
    Text(text = item.title)
}

[elijah-semyonov]

@EgorovDmitry Hi. Icon sets its SemanticsProperties.ContentDescription to "" with following in mind:

/**
 * Developer-set content description of the semantics node.
 *
 * If this is not set, accessibility services will present the [text][SemanticsProperties.Text] of
 * this node as the content.
 *
 * This typically should not be set directly by applications, because some screen readers will
 * cease presenting other relevant information when this property is present. This is intended
 * to be used via Foundation components which are inherently intractable to automatically
 * describe, such as Image, Icon, and Canvas.
 */
var SemanticsPropertyReceiver.contentDescription

Since the Row is clickable it merges its children, it merges all the semantics properties of its children on by-key basis according to per-key merge policy. Since you explicitly have contentDescription = "" it means the resulting semantics data will contain SemanticsProperties.ContentDescription with empty string inside that precedes SemanticsProperties.Text from Text(text = item.title), which is basically what the AX communicates to you.

In order to avoid that you need to either set contentDescription to null instead, or just not rely on automatic behavior at all and add Modifier.clearAndSetSemantics to your Row Modifier and set the expected semantic properties such as Text explicitly. They will be converted to accessibilityLabel communicated to AX services.

[SerdaOzun]

Not related and I might open an issue for this, but I noticed huge performance problems on macOS (but not on windows) which were apparently related to accessibility. Using the semantics(true) modifier solved that issue for me.

Happens in the context of FlowRow with thousands of clickable items inside it. So even though this issue is something else entirely, I wouldn't have known about the semantics modifier. Thanks :)

[okushnikov]

Please check the following ticket on YouTrack for follow-ups to this issue. GitHub issues will be closed in the coming weeks.