search

fabOnReact / react-native-notes

MIT License
0 stars 0 forks source link

Android: Position in Collection not supported by Flatlist, SectionList, VirtualizedList, or ScrollView #6

Closed fabOnReact closed 2 years ago

fabOnReact commented 2 years ago

Issue https://github.com/facebook/react-native/issues/30977 Pull Request https://github.com/facebook/react-native/pull/33180 Related https://github.com/facebook/react-native/pull/31666

fabOnReact commented 2 years ago
fabOnReact commented 2 years ago

<FlatList numColumns={undefined} /> should not trigger Runtime Error NoSuchKey exception columnCount when enabling TalkBack.

<FlatList numColumns={undefined} />

Expected Result: No Runtime Error

Actual Result: Enabling TalkBack triggers a Runtime Error NoSuchKeyException columnCount

CLICK TO OPEN RUNTIME ERROR

CLICK TO OPEN DETAILS

https://github.com/fabriziobertoglio1987/react-native/blob/e36f46a003a0c279d6d7e84bdfefc0b1a61d5cb7/Libraries/Lists/FlatList.js#L671-L675 https://github.com/fabriziobertoglio1987/react-native/blob/e36f46a003a0c279d6d7e84bdfefc0b1a61d5cb7/ReactAndroid/src/main/java/com/facebook/react/views/scroll/ReactHorizontalScrollView.java#L201 numColums default was removed with commit https://github.com/fabriziobertoglio1987/react-native-notes/commit/7d5895df4e3c57ebfa1c146cadc989497b006623

CLICK TO OPEN TESTS RESULTS - WITHOUT RUNTIME

CLICK TO OPEN TESTS RESULTS - WITH RUNTIME

Solved with https://github.com/fabriziobertoglio1987/react-native/commit/062cdcd107e150fce8be8978eee85ab13e0b2a14

fabOnReact commented 2 years ago

Evaluate Alternative Implementations

The ReactScrollView has a child node ReactViewGroup. It is not possible to retrieve the number of child from ReactViewGroup, because getChildCount was over-ridden and returns null.
https://github.com/fabriziobertoglio1987/react-native-notes/blob/5d3e3f8abd1cb9ece663db88137aded23e4c50d8/ReactAndroid/src/main/java/com/facebook/react/uimanager/ReactShadowNodeImpl.java#L256-L258

fabOnReact commented 2 years ago

Scenarios Previously Tested in https://github.com/facebook/react-native/pull/31666#issue-912733567 (VIDEO1 and VIDEO2)

New Scenarios

fabOnReact commented 2 years ago

CollectionItemInfo

Doubts and Tougths

These values are hardcoded. 1. What do they represent in the UI/functionality? 2. Which functionality we are not supporting? 3. When developers try to use unsupported functionality, do they receive a log message or alert that they are not supported? 4. Do we get a runtime or crash in the application because we did not handle/test these scenarios?

CollectionItemInfo in ReactNative

https://github.com/fabriziobertoglio1987/react-native-notes/blob/875076952dc8658d2cd61026411dbc24e05e90a3/Libraries/Lists/FlatList.js#L619 https://github.com/fabriziobertoglio1987/react-native-notes/blob/875076952dc8658d2cd61026411dbc24e05e90a3/Libraries/Lists/FlatList.js#L621 https://github.com/fabriziobertoglio1987/react-native-notes/blob/875076952dc8658d2cd61026411dbc24e05e90a3/Libraries/Lists/FlatList.js#L651-L653 https://github.com/fabriziobertoglio1987/react-native-notes/blob/875076952dc8658d2cd61026411dbc24e05e90a3/Libraries/Lists/FlatList.js#L622 The information are read by TalkBack from `AccessibilityNodeInfo`: - `rowSpan`, `columnSpan` will always be `1` - `heading` is always `false` https://github.com/fabriziobertoglio1987/react-native-notes/blob/875076952dc8658d2cd61026411dbc24e05e90a3/ReactAndroid/src/main/java/com/facebook/react/uimanager/ReactAccessibilityDelegate.java#L230-L238

CollectionItemInfo in AndroidX

https://github.com/androidx/androidx/blob/150112ca4e1e670fb4eb6756993f3a25df0a5da9/core/core/src/main/java/androidx/core/view/accessibility/AccessibilityNodeInfoCompat.java#L881 ```java /** * Class with information if a node is a collection item. *

* A collection item is contained in a collection, it starts at * a given row and column in the collection, and spans one or * more rows and columns. For example, a header of two related * table columns starts at the first row and the first column, * spans one row and two columns. *

*/ public static class CollectionItemInfoCompat { final Object mInfo; /** * Returns a cached instance if such is available otherwise a new one. * * @param rowIndex The row index at which the item is located. * @param rowSpan The number of rows the item spans. * @param columnIndex The column index at which the item is located. * @param columnSpan The number of columns the item spans. * @param heading Whether the item is a heading. This should be set to false and the newer * {@link AccessibilityNodeInfoCompat#setHeading(boolean)} used to identify * headings. * @param selected Whether the item is selected. * @return An instance. */ ```

Video Tests

Header

Behaviour without changing the values

fabOnReact commented 2 years ago
Columns in FlatList with multiple columns are not announced

- A blind user uses TalkBack to navigate items in the FlatList. - The FlatList has 2 columns and 6 rows (grid layout) - TalkBack announces each of the items and their column and row number **Expected Result**: Talkback announces: 1) `Marketplace row 1 column 1` 2) `FriendsList row 1 column 2` 3) `Groups row 2 column 1` 4) `Pages row 2 column 2` 5) `Fifth Item row 3 column 1` 6) `Sixth Item row 3 column 2` **Actual Result**: Talkback announces: 1) `Marketplace row 1` 2) `FriendsList` 3) `Groups row 2` 4) `Pages` 5) `Fifth Item row 3` 6) `Sixth Item` The TalkBack announcements are correct when using 1 column.

CLICK TO OPEN EXAMPLE

data ```Javascript const DATA = [ { id: 'bd7acbea-c1b1-46c2-aed5-3ad53abb28ba', title: 'Marketplace', }, { id: '3ac68afc-c605-48d3-a4f8-fbd91aa97f63', title: 'FriendsList', }, { id: '58694a0f-3da1-471f-bd96-145571e29d72', title: 'Groups', }, { id: 'bd7acbea-c1b1-46c2-aed5-3ad53abb8bbb', title: 'Pages', }, { id: '3ac68afc-c605-48d3-a4f8-fbd91aa97676', title: 'Fifth Item', }, { id: '58694a0f-3da1-471f-bd96-145571e27234', title: 'Sixth Item', }, ]; // and more items ``` JSX ```javascript <> item.id} numColumns={2} pagingEnabled={true} data={DATA} renderItem={renderItem} keyExtractor={item => item.toString()} /> ```

CLICK TO OPEN TESTS RESULTS - EXPECTED BEHAVIOUR

CLICK TO OPEN TESTS RESULTS - ACTUAL BEHAVIOUR

fabOnReact commented 2 years ago

SOLUTION FOR ISSUE https://github.com/fabriziobertoglio1987/react-native-notes/issues/6#issuecomment-1035689327

The issue is caused by this 2 lines. Commenting them and replacing them with a random integer number fixes the issue.

https://github.com/fabriziobertoglio1987/react-native-notes/blob/875076952dc8658d2cd61026411dbc24e05e90a3/ReactAndroid/src/main/java/com/facebook/react/views/scroll/ReactScrollView.java#L210-L211

CLICK TO OPEN VIDEO - NOT ANNOUNCING COLUMN

CLICK TO OPEN VIDEO - ANNOUNCING COLUMN

CLICK TO OPEN VIDEO - ANNOUNCING COLUMN AND ROW

CLICK TO OPEN VIDEO - SCROLL PAGE BEFORE

With 3 columns - Before the Change

CLICK TO OPEN VIDEO - SCROLL PAGE AFTER

With 3 columns - After the Change

CLICK TO OPEN MORE INFO

Removing the lines below (or replacing grid with list) would remove the row/column announcement. https://github.com/fabriziobertoglio1987/react-native-notes/blob/875076952dc8658d2cd61026411dbc24e05e90a3/ReactAndroid/src/main/java/com/facebook/react/uimanager/ReactAccessibilityDelegate.java#L409 https://github.com/fabriziobertoglio1987/react-native-notes/blob/875076952dc8658d2cd61026411dbc24e05e90a3/Libraries/Lists/FlatList.js#L698 more info at https://github.com/fabriziobertoglio1987/react-native-notes/issues/6#issuecomment-1034528419 The index is set correctly for columns 1 and 2, but TalkBack does not announce them. https://github.com/fabriziobertoglio1987/react-native-notes/blob/875076952dc8658d2cd61026411dbc24e05e90a3/Libraries/Lists/FlatList.js#L620 https://github.com/fabriziobertoglio1987/react-native-notes/blob/875076952dc8658d2cd61026411dbc24e05e90a3/ReactAndroid/src/main/java/com/facebook/react/uimanager/ReactAccessibilityDelegate.java#L237 The reason is that our Java Implementation of FlatList numColums does not use GridView, but simply uses ScrollView formatting data as a 2 columns table. https://github.com/fabriziobertoglio1987/react-native-notes/blob/875076952dc8658d2cd61026411dbc24e05e90a3/Libraries/Lists/FlatList.js#L492-L506 https://github.com/fabriziobertoglio1987/react-native-notes/blob/875076952dc8658d2cd61026411dbc24e05e90a3/ReactAndroid/src/main/java/com/facebook/react/views/scroll/ReactScrollView.java#L67 https://github.com/aosp-mirror/platform_frameworks_base/blob/1ac46f932ef88a8f96d652580d8105e361ffc842/core/java/android/widget/ScrollView.java#L56-L80 https://github.com/aosp-mirror/platform_frameworks_base/blob/1ac46f932ef88a8f96d652580d8105e361ffc842/core/java/android/widget/GridView.java#L54-L69 https://github.com/fabriziobertoglio1987/react-native-notes/blob/875076952dc8658d2cd61026411dbc24e05e90a3/ReactAndroid/src/main/java/com/facebook/react/uimanager/ReactAccessibilityDelegate.java#L146 Removing the above line: - TalkBack announces the item when scrolling pages - Rows are not announced anymore Adding the accessibilityRole `grid` adds the announcement for the rows.

CLICK TO OPEN TESTS RESULTS

fabOnReact commented 2 years ago

Test Cases

  1. ScreenReader announcing the wrong page ranges when scrolling up

The user scrolls up from the last page to the previous page. The page includes items from 10 to 21.

CLICK TO OPEN PAGES PREVIEW

| LAST PAGE | PREVIOUS PAGE | |:-------------------------:|:-------------------------:| | | |

Expected Result: Talkback announces the page Thirteen Item row 5 column 1 then says showing item 10 to 21 of 29

Actual Result: Scrolling Down: Same as Expected Result Scrolling Up: Talkback announces the page Thirteen Item row 5 column 1 then says showing item 10 to 19 of 29

CLICK TO OPEN SOURCECODE

```javascript item.id} numColumns={3} pagingEnabled={true} data={DATA} renderItem={renderItem} keyExtractor={item => item.toString()} /> ```

fabOnReact commented 2 years ago

INVESTIGATING ISSUE https://github.com/fabriziobertoglio1987/react-native-notes/issues/6#issuecomment-1038591574

Differences when scrolling with pagingEnabled without TalkBack and with TalkBack

**Without** Talkback (left): The screen displays items 10 to 19 of 29 **With** Talkback (right): The screen displays items 13 to 21 of 29 The ScrollView functionality with paging enabled is different between TalkBack and No-TalkBack. The method `getScrollDelta` (inspired by pr https://github.com/facebook/react-native/pull/25105) was designed not for TalkBack, but for a FlatList with `pagingEnabled={true}`. https://github.com/fabriziobertoglio1987/react-native-notes/blob/79777766ae8eec8594c1f0947f28a8965ef26653/ReactAndroid/src/main/java/com/facebook/react/views/scroll/ReactScrollView.java#L401-L405 | WITHOUT TALKBACK | WITH TALKBACK | |:-------------------------:|:-------------------------:| | | |

VIDEO OF THE ISSUE

CLICK TO OPEN OTHER NOTES

The issue is caused by `isPartiallyScrolledInView` not computing correctly visible childs of ScrollView. https://github.com/fabriziobertoglio1987/react-native-notes/blob/79777766ae8eec8594c1f0947f28a8965ef26653/ReactAndroid/src/main/java/com/facebook/react/views/scroll/ReactScrollView.java#L158 https://github.com/fabriziobertoglio1987/react-native-notes/blob/79777766ae8eec8594c1f0947f28a8965ef26653/ReactAndroid/src/main/java/com/facebook/react/views/scroll/ReactScrollView.java#L407-L412 https://stackoverflow.com/a/4629167/7295772

The issue seems caused by isVisible

Debugging isPartiallyScrolledInView

fabOnReact commented 2 years ago

SOLUTION FOR ISSUE https://github.com/fabriziobertoglio1987/react-native-notes/issues/6#issuecomment-1038591574

Video of Debugging Values return in isMostlyScrolledInView for Item 10 and 20 while scrolling down

Video of Debugging Values return in isMostlyScrolledInView for Item 10 and 20 while scrolling up

Testing and troubleshooting the implementation

https://github.com/fabriziobertoglio1987/react-native/commit/048c5b9a922b628841b1d5f81051f9de4be7fcb3 and https://github.com/fabriziobertoglio1987/react-native/commit/602e81a7401f5851edcd9147f76ba387bc93a9bd

fabOnReact commented 2 years ago

Test Cases

  1. Not announcing correctly the first page items range

TalkBack announces the first page

Expected Result: Showing items 1 to 19 of 29

Actual Result: Showing items 10 of 29

CLICK TO OPEN VIDEO - NOT ANNOUNCING FIRST ITEM FIRST PAGE

CLICK TO OPEN VIDEO - ANNOUNCING FIRST ITEM FIRST PAGE

Solved with https://github.com/fabriziobertoglio1987/react-native-notes/commit/cbe286d4a84ae555b7c2dd54f032a99e2f628811

fabOnReact commented 2 years ago

Using height instead of width

Solves https://github.com/fabriziobertoglio1987/react-native-notes/issues/6#issuecomment-1038591574 https://github.com/fabriziobertoglio1987/react-native-notes/issues/6#issuecomment-1039991416. The method from ReactHorizontalScrollView scrolls in a different direction. computeScrollDeltaToGetChildRectOnScreen is inherited from ScrollView. Video of the tests in https://github.com/fabriziobertoglio1987/react-native-notes/issues/6#issuecomment-1040937316.

  private boolean isPartiallyScrolledInView(View descendent) {
    int scrollDelta = getScrollDelta(descendent);
    descendent.getDrawingRect(mTempRect);
    return scrollDelta != 0 && Math.abs(scrollDelta) < mTempRect.height();
  }
fabOnReact commented 2 years ago

Using a specific accessibility event

https://developer.android.com/reference/android/view/accessibility/AccessibilityEvent

if (event.getEventType() == AccessibilityEvent.TYPE_WINDOW_CONTENT_CHANGED) {
    // event.setItemCount(10)
}
fabOnReact commented 2 years ago

Loop through each column

Solved https://github.com/fabriziobertoglio1987/react-native-notes/issues/6#issuecomment-1038591574 https://github.com/fabriziobertoglio1987/react-native-notes/issues/6#issuecomment-1039991416. Each View has numColumns number child views (for each column). The last 2 columns of each last row are not announced.

https://github.com/fabriziobertoglio1987/react-native-notes/blob/79777766ae8eec8594c1f0947f28a8965ef26653/ReactAndroid/src/main/java/com/facebook/react/views/scroll/ReactScrollView.java#L171

needs to be changed somehow 

if (childCount > 0 && accessibilityCollectionItemInfo == null) {
  for(int child = 0; child < childCount; child++) {
    View nestedNextChild =  ((ViewGroup) nextChild).getChildAt(child);
    if (nestedNextChild != null) {
      // logic
    }
  }
}

Implemented with commit https://github.com/fabriziobertoglio1987/react-native-notes/commit/8dc7fa9539a66838d39af593140f61ca7b3e3a21 and https://github.com/fabriziobertoglio1987/react-native-notes/commit/cbe286d4a84ae555b7c2dd54f032a99e2f628811 Test included in https://github.com/fabriziobertoglio1987/react-native-notes/issues/6#issuecomment-1040937316

fabOnReact commented 2 years ago

Not announcing correctly the last page (partial scroll)

Solves https://github.com/fabriziobertoglio1987/react-native-notes/issues/6#issuecomment-1038591574 https://github.com/fabriziobertoglio1987/react-native-notes/issues/6#issuecomment-1039991416. The issue was caused by ScrollDelta with value 0 (solved with https://github.com/fabriziobertoglio1987/react-native/commit/ee23f451bc428431e9b29e26e5b00c7580fa3e78).

fabOnReact commented 2 years ago

TalkBack announces pages when scrolling down a GridView

TalkBack announces showing item 10 to 21 of 29 when scrolling pages in a GridView.

https://github.com/fabriziobertoglio1987/MyApplication/commit/0bfa67e43ce10c67adfedab2dc4af948e9ad8aee

fabOnReact commented 2 years ago

https://docs.google.com/document/d/1Y6Ze9KflkPUVQHxFuWYhx4KsegJEBYoldXITVyjcz58/edit?disco=AAAAUu3C_L4

fabOnReact commented 2 years ago
use mock values to set nodeInfo.setCollectionItemInfo

use mock values to set `nodeInfo.setCollectionItemInfo` (each node of the collection) and hear reading aloud each cell row and column, as done with the header role https://github.com/fabriziobertoglio1987/react-native-notes/issues/6#issuecomment-1035689327 https://developer.android.com/reference/android/view/accessibility/AccessibilityNodeInfo.CollectionItemInfo >A collection item is contained in a collection, it starts at a given row and column in the collection, and spans one or more rows and columns. For example, a header of two related table columns starts at the first row and the first column, spans one row and two columns. https://github.com/fabriziobertoglio1987/react-native-notes/blob/ee23f451bc428431e9b29e26e5b00c7580fa3e78/ReactAndroid/src/main/java/com/facebook/react/uimanager/ReactAccessibilityDelegate.java#L437-L439

fabOnReact commented 2 years ago
debug grid example

Understand how they calculate visible children's https://github.com/aosp-mirror/platform_frameworks_base/blob/1ac46f932ef88a8f96d652580d8105e361ffc842/core/java/android/widget/AdapterView.java#L664-L682

fabOnReact commented 2 years ago
use isVisibleToUser to detect visible node

use [isVisibleToUser](https://developer.android.com/reference/androidx/core/view/accessibility/AccessibilityNodeInfoCompat#isVisibleToUser()) to detect which nodes are visible https://github.com/fabriziobertoglio1987/react-native-notes/blob/ee23f451bc428431e9b29e26e5b00c7580fa3e78/ReactAndroid/src/main/java/com/facebook/react/uimanager/ReactAccessibilityDelegate.java#L437-L439

fabOnReact commented 2 years ago
move logic from onInitializeAccessibilityInfo to ReactAccessibilityDelegate

Move logic to ReactAccessibilityDelegate and test scenario https://github.com/fabriziobertoglio1987/react-native-notes/issues/6#issuecomment-1035689327 https://github.com/fabriziobertoglio1987/react-native-notes/blob/ee23f451bc428431e9b29e26e5b00c7580fa3e78/ReactAndroid/src/main/java/com/facebook/react/views/scroll/ReactScrollView.java#L201-L229 https://github.com/fabriziobertoglio1987/react-native-notes/blob/ee23f451bc428431e9b29e26e5b00c7580fa3e78/ReactAndroid/src/main/java/com/facebook/react/uimanager/ReactAccessibilityDelegate.java#L437-L439

move logic from onInitializeAccessibilityEvent to ReactAccessibilityDelegate

replace logic in `ReactScrollView` method `onInitializeAccessibilityEvent` with https://github.com/aosp-mirror/platform_frameworks_base/blob/1ac46f932ef88a8f96d652580d8105e361ffc842/core/java/android/widget/AdapterView.java#L1018-L1030 https://github.com/fabriziobertoglio1987/react-native-notes/blob/ee23f451bc428431e9b29e26e5b00c7580fa3e78/ReactAndroid/src/main/java/com/facebook/react/views/scroll/ReactScrollView.java#L136-L199

fabOnReact commented 2 years ago
fabOnReact commented 2 years ago

TalkBack announces pages and cells with Horizontal Flatlist in the Paper Renderer

Expected/Actual Result: Talkback announces the selected item row and column changes: 1) 1 Item one of 29 2) 2 Item two of 29

Expected/Actual Result: Talkback announces the pages when scrolling down:

Thirteen Item row 5 column 1 then says showing item 2 to 4 of 29
CLICK TO OPEN SOURCECODE

https://github.com/fabriziobertoglio1987/ReactNativeAwesomeProject/blob/af0d0baa95f4520cd05d61031a69a66ec8521bf0/App.js ```jsx item.toString()} /> ``` ```javascript const Item = ({title}) => ( {title} ); const renderFlatList = ({item}) => ( ); ```

data used in flatlist

```javascript const DATA = [ { id: '1', title: '1 Item', }, { id: '2', title: '2 Item', }, { id: '3', title: '3 Item', }, { id: '4', title: '4 Item', }, { id: '5', title: 'Fifth Item', }, { id: '6', title: 'Sixth Item', }, { id: '7', title: 'Seven Item', }, { id: '8', title: 'Eight Item', }, { id: '9', title: 'Nine Item', }, { id: '10', title: 'Ten Item', }, { id: '11', title: 'Eleven Item', }, { id: '12', title: 'Twelve Item', }, { id: '13', title: 'Thirdteen Item', }, { id: '14', title: 'Fourteen Item', }, { id: '15', title: 'Fifthteen Item', }, { id: '16', title: 'Sixteen Item', }, { id: '17', title: '17 Item', }, { id: '18', title: '18 Item', }, { id: '19', title: '19 Item', }, { id: '20', title: '20 Item', }, { id: '21', title: '21 Item', }, { id: '22', title: '22 Item', }, { id: '21', title: '23 Item', }, { id: '22', title: '24 Item', }, { id: '22', title: '25 Item', }, { id: '23', title: '26 Item', }, { id: '24', title: '27 Item', }, { id: '25', title: '28 Item', }, { id: '26', title: '29 Item', }, ]; ```

CLICK TO OPEN VIDEO TESTS

fabOnReact commented 2 years ago

TalkBack announces pages and cells with Vertical Flatlist in the Paper Renderer

Expected/Actual Result: Talkback announces the selected item row and column changes: 1) 2 Item column 2 2) 4 Item row two column 1 3) 11 Item column 2

Expected/Actual Result: Talkback announces the pages when scrolling down:

Showing Item 10 to 19 of 29
CLICK TO OPEN SOURCECODE

https://github.com/fabriziobertoglio1987/ReactNativeAwesomeProject/commit/016921e96940693fbb2ed195c5e8d051330462ff ```javascript class App extends React.Component { constructor(props) { super(props); } header = () => { return ( POSTS ); }; render() { return ( item.id} pagingEnabled={false} data={DATA} renderItem={renderItem} keyExtractor={item => item.toString()} /> ); } } const renderItem = ({item, key}) => ( {item.title} ); ``` ```javascript const Item = ({title}) => ( {title} ); const renderFlatList = ({item}) => ( ); ```

data used in flatlist

```javascript const DATA = [ { id: '1', title: '1 Item', }, { id: '2', title: '2 Item', }, { id: '3', title: '3 Item', }, { id: '4', title: '4 Item', }, { id: '5', title: 'Fifth Item', }, { id: '6', title: 'Sixth Item', }, { id: '7', title: 'Seven Item', }, { id: '8', title: 'Eight Item', }, { id: '9', title: 'Nine Item', }, { id: '10', title: 'Ten Item', }, { id: '11', title: 'Eleven Item', }, { id: '12', title: 'Twelve Item', }, { id: '13', title: 'Thirdteen Item', }, { id: '14', title: 'Fourteen Item', }, { id: '15', title: 'Fifthteen Item', }, { id: '16', title: 'Sixteen Item', }, { id: '17', title: '17 Item', }, { id: '18', title: '18 Item', }, { id: '19', title: '19 Item', }, { id: '20', title: '20 Item', }, { id: '21', title: '21 Item', }, { id: '22', title: '22 Item', }, { id: '21', title: '23 Item', }, { id: '22', title: '24 Item', }, { id: '22', title: '25 Item', }, { id: '23', title: '26 Item', }, { id: '24', title: '27 Item', }, { id: '25', title: '28 Item', }, { id: '26', title: '29 Item', }, ]; ```

CLICK TO OPEN VIDEO TESTS

fabOnReact commented 2 years ago

Summary

This issue fixes https://github.com/facebook/react-native/issues/30977. The Pull Request was previously published by intergalacticspacehighway with https://github.com/facebook/react-native/pull/31666.

The solution consists of:

  1. Adding Javascript logic in the FlatList component to add accessibility information (row and column position) for each FlatList cell. The information is saved on the native side in the AccessibilityNodeInfo and announced by TalkBack when changing row, column, or page (video example).

  2. Adding Java logic in ReactScrollView.java and HorizontalScrollView to announce pages with TalkBack when scrolling up/down. The missing AOSP logic in ScrollView.java (see also the GridView example) is responsible for announcing Page Scrolling with TalkBack.

Relevant discussions https://github.com/fabriziobertoglio1987/react-native-notes/issues/6

Changelog

[Android] [Added] - Accessibility announcement for list and grid in FlatList

Test Plan

1. TalkBack announces pages and cells with Horizontal Flatlist in the Paper Renderer (link) 2. TalkBack announces pages and cells with Vertical Flatlist in the Paper Renderer (link) 3. FlatList numColumns={undefined} Should not trigger Runtime Error NoSuchKey exception columnCount when enabling TalkBack. (link) 4. TalkBack announces pages and cells with Nested Horizontal Flatlist in the rn-tester app (link)

fabOnReact commented 2 years ago

TalkBack announces pages and cells with Nested Horizontal Flatlist in the rn-tester app

Expected/Actual Result: Talkback announces the selected item row and column changes: 1) 2 Item column 2 2) 4 Item row two column 1 3) 11 Item column 2

Expected/Actual Result: Talkback announces the pages when scrolling down:

Showing Item 10 to 19 of 29
CLICK TO OPEN VIDEO TESTS

fabOnReact commented 2 years ago

https://github.com/facebook/react-native/pull/33180#discussion_r820860260

Screenshots of the two scenarios (with and without {flex: 1})

| without `{flex: 1}` | with `{flex: 1}` | |:-------------------------:|:-------------------------:| | | |

I moved this logic to VirtualizedList with commits move accessibilityCollectionItem logic to VirtList, Move Logic to virtualized list and removing container View in renderItem callback removing the View container around the cell.

The callback renderItem now has the accessibilityCollectionItem.

Example of the implementation with FlatList

https://github.com/facebook/react-native/blob/c0893d20470e5971eaee5ccc74196dbe8ec84770/packages/rn-tester/js/examples/FlatList/FlatList-multiColumn.js#L143-L156

I added the prop type accessibilityCollectionItem to the View Component, which is used in this case as a wrapper around each FlatList/SectionList cell.

accessibilityCollectionItem in ViewPropTypes

https://github.com/facebook/react-native/blob/c0893d20470e5971eaee5ccc74196dbe8ec84770/Libraries/Components/View/ViewPropTypes.js#L444-L459

The previous solution was adding a wrapper View Component around each cell. The wrapper in VirtualizedList spans for the entire row (explanation in this post) and does not work with multicolumn flatlist.

I would like to receive your feedback on this solution before investing further time. Is this the solution you had in mind? I can revert this changes or improve it based on your feedback. Thanks

fabOnReact commented 2 years ago

https://github.com/facebook/react-native/pull/33180#discussion_r820864643

Why is this $FlowFixMe needed?

The original PR had the following test failing.

Circleci test failing log

``` Error ------------------------------------------------------------------------------- Libraries/Lists/FlatList.js:657:14 Cannot create `View` element because property `accessibilityCollectionItem` is missing in `ViewProps` [1] but exists in props [2]. [prop-missing] Libraries/Lists/FlatList.js:657:14 657| 661| {renderer(info)} 662| ------^ [2] ```

The failure is caused by the missing prop accessibilityCollectionItem in FlatList.js.

My first solution was adding the accessibilityCollectionItem to the ViewProps. My second solution was using the $FlowFixMe[prop-missing]. I decided to use this solution for the following reasons:

We may consider moving this to internal_prop as done for example in internal_analyticTag.

fabOnReact commented 2 years ago

https://github.com/facebook/react-native/pull/33180#discussion_r820866420

Why is this $FlowFixMe necessary?

To fix the following test failing in CircleCI.

Circleci failing test log

```javascript Error ------------------------------------------------------------------------------- Libraries/Lists/FlatList.js:672:36 Cannot call `this._getItemCount` with `this.props.data` bound to `data` because read-only array type [1] is incompatible with array type [2]. [incompatible-call] Libraries/Lists/FlatList.js:672:36 672| rowCount: this._getItemCount(this.props.data), ^^^^^^^^^^^^^^^ References: Libraries/Lists/FlatList.js:35:10 35| data: ?$ReadOnlyArray, ^^^^^^^^^^^^^^^^^^^^^ [1] Libraries/Lists/FlatList.js:508:27 508| _getItemCount = (data: ?Array): number => { ^^^^^^^^^^^^ [2] Error ------------------------------------------------------------------------------- Libraries/Lists/FlatList.js:696:2 ```

The method _getItemCount(data) can not be called from FlatList with parameter this.props.data.

For simplicity, data is just a plain array. If you want to use something else, like an immutable list, use the underlying VirtualizedList directly.

data: ?$ReadOnlyArray<ItemT>,

An alternative solution I previously implemented in commit e36b2610

   _getAccessibilityCollection = () => {
     const _getItemCountInternal = (data: any): number => {
       return this._getItemCount(data);
     };
fabOnReact commented 2 years ago
fabOnReact commented 2 years ago
moving _getAccessibilityCollection() to virtualized list

https://github.com/fabriziobertoglio1987/react-native-notes/blob/ce18d88eb12e50e40b5e6c02bc08da74a5ed5392/Libraries/Lists/FlatList.js#L685 move to https://github.com/fabriziobertoglio1987/react-native-notes/blob/ce18d88eb12e50e40b5e6c02bc08da74a5ed5392/Libraries/Lists/VirtualizedList.js#L2057-L2079

fabOnReact commented 2 years ago
move accessibilityCollectionItem logic to VirtualizedList

https://github.com/fabriziobertoglio1987/react-native-notes/blob/ce18d88eb12e50e40b5e6c02bc08da74a5ed5392/Libraries/Lists/FlatList.js#L615-L640 move to https://github.com/fabriziobertoglio1987/react-native-notes/blob/ce18d88eb12e50e40b5e6c02bc08da74a5ed5392/Libraries/Lists/VirtualizedList.js#L1983-L2017 The current View container, but represents entire row (2 cells below) https://github.com/fabriziobertoglio1987/react-native-notes/blob/f49caa490eb2378466bfa3eeb672aa6eab5b65b7/Libraries/Lists/VirtualizedList.js#L2073