flutter / flutter

Flutter makes it easy and fast to build beautiful apps for mobile and beyond
https://flutter.dev
BSD 3-Clause "New" or "Revised" License
165.19k stars 27.26k forks source link

VoiceOver doesn't recognise scroll using GestureDetector #83699

Open Mateis opened 3 years ago

Mateis commented 3 years ago

Steps to Reproduce

  1. Run flutter create bug.
  2. Update the files as follows:

Replace the default child of Center() with a GestureDetector

Center(
        // Center is a layout widget. It takes a single child and positions it
        // in the middle of the parent.
        child: GestureDetector(
          onVerticalDragEnd: (endDetails) {
            double velocity = endDetails.primaryVelocity;
            if        (velocity < 0) {
              print('GESTURE -- Swipe Up');
            } else if (velocity > 0) {
              print('GESTURE -- Swipe Down');
            }
          },
        )
      ),
  1. Run the app (on physical device)
  2. Turn on VoiceOver
  3. Do a three finger swipe in the 'bug' app.

Expected results: Without VoiceOver activated, the app registers scroll action with a one finger swipe. With VoicerOver activated, the app registers scroll actions with a three finger swipe.

Three finger swipe is how users with VoiceOver activated scroll.

Actual results: As expected, without VoiceOver accessibility turned on, the scrolling using (one finger) swipe works. Unexpected, with VoiceOver on, the GestureDetector widget does not register a swipe action and users are not able to scroll using a three finger swipe.

``` flutter analyze Analyzing voiceoverscrolltest... info • The value of the field '_counter' isn't used • lib/main.dart:53:7 • unused_field 1 issue found. (ran in 3.8s) ``` ``` flutter doctor -v [✓] Flutter (Channel stable, 1.22.5, on Mac OS X 10.15.7 19H524 darwin-x64, locale en-NL) • Flutter version 1.22.5 at /Users/Mat/Documents/development/flutter • Framework revision 7891006299 (6 months ago), 2020-12-10 11:54:40 -0800 • Engine revision ae90085a84 • Dart version 2.10.4 [✗] Android toolchain - develop for Android devices ✗ Unable to locate Android SDK. Install Android Studio from: https://developer.android.com/studio/index.html On first launch it will assist you in installing the Android SDK components. (or visit https://flutter.dev/docs/get-started/install/macos#android-setup for detailed instructions). If the Android SDK has been installed to a custom location, set ANDROID_SDK_ROOT to that location. You may also want to add it to your PATH environment variable. [✓] Xcode - develop for iOS and macOS (Xcode 12.3) • Xcode at /Applications/Xcode.app/Contents/Developer • Xcode 12.3, Build version 12C33 • CocoaPods version 1.10.1 [!] Android Studio (not installed) • Android Studio not found; download from https://developer.android.com/studio/index.html (or visit https://flutter.dev/docs/get-started/install/macos#android-setup for detailed instructions). [✓] VS Code (version 1.56.2) • VS Code at /Applications/Visual Studio Code.app/Contents • Flutter extension version 3.22.0 [✓] Connected device (1 available) ```
TahaTesser commented 3 years ago

Hi @Mateis VoiceOver doesn't even seem to focus on GestureDetectorr hence it won't recognize any gestures, Is that the same for you?

https://user-images.githubusercontent.com/48603081/120312668-b1b82e80-c2e1-11eb-88af-9ddf9f76989e.mov

minimal code sample ```dart import 'package:flutter/material.dart'; void main() => runApp(MyApp()); class MyApp extends StatelessWidget { @override Widget build(BuildContext context) { return MaterialApp( title: 'Material App', theme: ThemeData.dark(), home: Home(), ); } } class Home extends StatelessWidget { @override Widget build(BuildContext context) { return Scaffold( appBar: AppBar( title: Text('Material App Bar'), ), body: GestureDetector( onVerticalDragEnd: (endDetails) { double velocity = endDetails.primaryVelocity; if (velocity < 0) { print('GESTURE -- Swipe Up'); } else if (velocity > 0) { print('GESTURE -- Swipe Down'); } }, child: FlutterLogo(size: 300,), ), floatingActionButton: FloatingActionButton( child: Icon(Icons.add), onPressed: () {}, ), ); } } ```
flutter doctor -v ```console [✓] Flutter (Channel stable, 2.2.1, on macOS 11.4 20F71 darwin-x64, locale en-GB) • Flutter version 2.2.1 at /Users/tahatesser/Code/flutter_stable • Framework revision 02c026b03c (5 days ago), 2021-05-27 12:24:44 -0700 • Engine revision 0fdb562ac8 • Dart version 2.13.1 [✓] Android toolchain - develop for Android devices (Android SDK version 30.0.3) • Android SDK at /Volumes/Extreme/SDK • Platform android-30, build-tools 30.0.3 • ANDROID_HOME = /Volumes/Extreme/SDK • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264) • All Android licenses accepted. [✓] Xcode - develop for iOS and macOS • Xcode at /Volumes/Extreme/Xcode.app/Contents/Developer • Xcode 12.5, Build version 12E262 • CocoaPods version 1.10.1 [✓] Chrome - develop for the web • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome [✓] Android Studio (version 4.2) • Android Studio at /Applications/Android Studio.app/Contents • Flutter plugin can be installed from: 🔨 https://plugins.jetbrains.com/plugin/9212-flutter • Dart plugin can be installed from: 🔨 https://plugins.jetbrains.com/plugin/6351-dart • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264) [✓] VS Code (version 1.56.2) • VS Code at /Applications/Visual Studio Code.app/Contents • Flutter extension version 3.22.0 [✓] Connected device (4 available) • Taha’s iPhone (mobile) • 00008020-001059882212002E • ios • iOS 14.6 • iPhone 12 Pro Max (mobile) • 516FAD15-C23C-4952-943C-0612DAAC1643 • ios • com.apple.CoreSimulator.SimRuntime.iOS-14-5 (simulator) • macOS (desktop) • macos • darwin-x64 • macOS 11.4 20F71 darwin-x64 • Chrome (web) • chrome • web-javascript • Google Chrome 91.0.4472.77 • No issues found! ```
Mateis commented 3 years ago

Yes, I have the same behaviour. It doesn't select.

Maybe a separate Semantic wrapping widget is needed (on the gesture detector or Center widget)?

Mateis commented 3 years ago

Yes, the following code made it work as expected:

import 'package:flutter/material.dart';

void main() {
  runApp(MyApp());
}

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        primarySwatch: Colors.blue,
        visualDensity: VisualDensity.adaptivePlatformDensity,
      ),
      home: MyHomePage(title: 'Flutter Demo Home Page'),
    );
  }
}

class MyHomePage extends StatefulWidget {
  MyHomePage({Key key, this.title}) : super(key: key);

  final String title;

  @override
  _MyHomePageState createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {

  scrollDown(){
    print(' scroll down');
  }
  scrollUp(){
    print(' scroll up');
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text(widget.title),
      ),
      body: Center(
        child: 
        Semantics(
          focusable: true,
          onScrollDown: scrollDown,
          onScrollUp: scrollUp,
          child:
        GestureDetector(
          onVerticalDragEnd: (endDetails) {
            double velocity = endDetails.primaryVelocity;
            if        (velocity < 0) {
              print('GESTURE -- Swipe Up');
            } else if (velocity > 0) {
              print('GESTURE -- Swipe Down');
            }
          },
        )
        )
      ),
      floatingActionButton: FloatingActionButton(
        onPressed: () {},
        tooltip: 'Increment',
        child: Icon(Icons.add),
      ), // This trailing comma makes auto-formatting nicer for build methods.
    );
  }
}

Hence, I had to wrap the GestureWidget with a Semantics widget and set the focus, OnScrollDown and OnScrollUp parameters.

       Semantics(
          focusable: true,
          onScrollDown: scrollDown,
          onScrollUp: scrollUp,
TahaTesser commented 3 years ago

Hi @Mateis I think your right Semantics might be required, see https://github.com/flutter/flutter/issues/49285 Even tho it prints, events are not announced, Voiceover doesn't say anything when I can scroll. GestureDetector uses Raw GestureDetector which has semantics https://api.flutter.dev/flutter/widgets/RawGestureDetector/semantics.html. Shouldn't it be sufficient?

I think Semantics shouldn't require unless it is universal to all gesture widgets, working as design but what about the announcement, it doesn't solve that

Mateis commented 3 years ago

Hmm yeah makes sense if you put it like that. On Android it seems to work fine without extra Semantics Widget, so that's odd.

TahaTesser commented 3 years ago

Hi @Mateis I can confirm it works out Semantics widget on Android. Yet no announcements tho

stable master

Check flutter doctor -v outputs for each channel below

flutter doctor -v ```console [✓] Flutter (Channel stable, 2.2.1, on macOS 11.4 20F71 darwin-x64, locale en-GB) • Flutter version 2.2.1 at /Users/tahatesser/Code/flutter_stable • Framework revision 02c026b03c (2 weeks ago), 2021-05-27 12:24:44 -0700 • Engine revision 0fdb562ac8 • Dart version 2.13.1 [✓] Android toolchain - develop for Android devices (Android SDK version 30.0.3) • Android SDK at /Volumes/Extreme/SDK • Platform android-30, build-tools 30.0.3 • ANDROID_HOME = /Volumes/Extreme/SDK • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264) • All Android licenses accepted. [✓] Xcode - develop for iOS and macOS • Xcode at /Volumes/Extreme/Xcode-beta.app/Contents/Developer • Xcode 13.0, Build version 13A5154h • CocoaPods version 1.10.1 [✓] Chrome - develop for the web • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome [✓] Android Studio (version 4.2) • Android Studio at /Applications/Android Studio.app/Contents • Flutter plugin can be installed from: 🔨 https://plugins.jetbrains.com/plugin/9212-flutter • Dart plugin can be installed from: 🔨 https://plugins.jetbrains.com/plugin/6351-dart • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264) [✓] VS Code (version 1.56.2) • VS Code at /Applications/Visual Studio Code.app/Contents • Flutter extension version 3.23.0 [✓] Connected device (3 available) • IN2011 (mobile) • c9d8ee0c • android-arm64 • Android 11 (API 30) • macOS (desktop) • macos • darwin-x64 • macOS 11.4 20F71 darwin-x64 • Chrome (web) • chrome • web-javascript • Google Chrome 91.0.4472.77 • No issues found! ``` ```console [✓] Flutter (Channel master, 2.3.0-17.0.pre.295, on macOS 11.4 20F71 darwin-x64, locale en-GB) • Flutter version 2.3.0-17.0.pre.295 at /Users/tahatesser/Code/flutter_master • Upstream repository https://github.com/flutter/flutter.git • Framework revision f289aa19e7 (2 hours ago), 2021-06-10 02:29:02 -0400 • Engine revision ad6c83164e • Dart version 2.14.0 (build 2.14.0-194.0.dev) [✓] Android toolchain - develop for Android devices (Android SDK version 30.0.3) • Android SDK at /Volumes/Extreme/SDK • Platform android-30, build-tools 30.0.3 • ANDROID_HOME = /Volumes/Extreme/SDK • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264) • All Android licenses accepted. [✓] Xcode - develop for iOS and macOS • Xcode at /Volumes/Extreme/Xcode-beta.app/Contents/Developer • Xcode 13.0, Build version 13A5154h • CocoaPods version 1.10.1 [✓] Chrome - develop for the web • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome [✓] Android Studio (version 4.2) • Android Studio at /Applications/Android Studio.app/Contents • Flutter plugin can be installed from: 🔨 https://plugins.jetbrains.com/plugin/9212-flutter • Dart plugin can be installed from: 🔨 https://plugins.jetbrains.com/plugin/6351-dart • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264) [✓] IntelliJ IDEA Community Edition (version 2021.1.2) • IntelliJ at /Volumes/Extreme/IntelliJ IDEA CE.app • Flutter plugin can be installed from: 🔨 https://plugins.jetbrains.com/plugin/9212-flutter • Dart plugin can be installed from: 🔨 https://plugins.jetbrains.com/plugin/6351-dart [✓] VS Code (version 1.56.2) • VS Code at /Applications/Visual Studio Code.app/Contents • Flutter extension version 3.23.0 [✓] Connected device (3 available) • IN2011 (mobile) • c9d8ee0c • android-arm64 • Android 11 (API 30) • macOS (desktop) • macos • darwin-x64 • macOS 11.4 20F71 darwin-x64 • Chrome (web) • chrome • web-javascript • Google Chrome 91.0.4472.77 • No issues found! ```

✅ : No Issue ❌: Issue reproduced

TahaTesser commented 3 years ago

@Mateis Let's keep the issue for iOS as it requires an extra Semantics widget

For the announcements on both platforms, we can track in a new issue

cc: @chunhtai

chunhtai commented 3 years ago

@Mateis Adding the additional semantics node makes sense to me, the gesture recognizer only make it draggable, but not necessary scrollable. drag can be any custom action, but scrolling is a very specific term.

TahaTesser commented 3 years ago

@chunhtai

Do you know why it doesn't require on Android? and how we can make it announce, Is that a bug in the first?