[Enhancement] Reduce the number of swipes a TalkBack user needs for a toot

[MarcoZehe]

Problem

When the TalkBack screen reader is on, a blind user can swipe left or right to walk sequentially through the items on the screen.

When reading toots or other items, right now a TalkBack user needs several swipes to walk through a toot. The standard items are:

1. Profile name, and CW text if applicable.
2. Profile button.
3. Tot text, or Show More button if hidden behind CW.
4. If 3 was a CW enhancer, toot text.
5. Reply Button.
6. Boost button.
7. Favorite Button.
8. More Options button.

If there is one image, between items 4 and 5, two more are inserted, plus one more for each aditional image.

That makes a minimum of 7 or 8, and a maximum of 10 swipes for one toot, with at least half of the items being repetitive information like the buttons etc. This makes browsing toots very inefficient for a blind user.

Other clients, for example for Twitter, have adopted a model where each item only consists of one swipable item, and the actual actions are then provided as Accessibility.AccessibilityAction items. That reduces the number of swipes per item to 1, and the actions can be taken if needed or wanted, or one can just continue with the next toot.

Proposed solution

To do this in Tusky, the following would need to happen:

1. All items except one would need to be marked Accessibility.AccessibilityNodeInfo.SetImportantForAccessibility false, so TalkBack no longer swipes to them.
2. Set the contentDescription to a string of text containing all relevant information from most to least important. They should be separated by a comma, so the speech synthesizer inserts a slight pause between the items, but not too much of a pause, like it would do if a period was used instead. Alternatively, a semicolon ; could also be tried. A proposed order would be:
   1. Profile screen name, but not the Mastodon username.
   2. The toot text, or CW if present, plus a hint that this is a CW.
   3. If this is a boost, something like "Boosted by xyz".
   4. The date this was composed.
   5. The full Mastodon username.
   6. If there are images attached and these have descriptions, add them one by one to the end.
3. Add AccessibilityNodeInfo.AccessibilityAction classes for all the supported actions. Those actions would also be in the order "most likely" to "least likely", as they are presented top to bottom in the local context menu of TalkBack. The proposed order is:
   1. Default action, would be either open the toot thread, or expand the CW. The CW would then need to be replaced with the real toot text on the contentDescription.
   2. Reply
   3. Boost
   4. Favorite
   5. Open profile
   6. An action for each image if one can normally also click on them (makes them full-screen maybe?).
4. A special case is for the Notifications, because there are several different types. The contentDescription for each of those items needs to be crafted accordingly, like for boost, favorite (these are similar), replies, follows etc.

If these enhancements are made, that would make Tusky way more efficient to use for a blind user.

---

• Tusky Version: 4.1

• Android Version: Android 9 (Pie)

• Android Device: Google Pixel 3

• Mastodon instance (if applicable): Any.

• [x] I searched or browsed the repo’s other issues to ensure this is not a duplicate.

[charlag]

@MarcoZehe hey, Marco! I have a question if you don't mind. I half-implemented it but I have a question about links/mentions. Normally you can click on them but if we interpret the whole status as one item then you cannot really select it. I noticed that when you can focus text, there's an action with all the links. We could allow focusing text separately when the status is opened on a separate screen or maybe there's some other way but I don't see how they did it yet.

[MarcoZehe]

I recently saw something similar being implemented in the IRCCloud Android client. In particular, this commit might help show how this is accomplished in the contentDescription.

[davidbtc2009]

Shouldnt this issue be closed?