Screen readers (e.g. VoiceOver) are not working for Flutter websites and macOS apps

nilsreichardt commented 1 year ago

Description

Hey! 👋 I'm trying to get the macOS VoiceOver working for Flutter websites and Flutter macOS apps. However, it's not working. It's working for all other macOS apps or all other websites except that one created with Flutter.

https://user-images.githubusercontent.com/24459435/201398484-4aa5fc1b-02a8-4e08-86b5-482a783b035b.mp4

I tested the https://gallery.flutter.dev for web and pulled the https://github.com/flutter/gallery (11.11.2022) repository to build & test the macOS (with v3.5.0-10.0.pre.121).

Steps to reproduce

Go to https://gallery.flutter.dev
Enable VoiceOver in your macOS settings
Navigate with the tab key

What I would expect

I would expect that VoiceOver is working like it shown in this video by Flutter: https://www.youtube.com/watch?v=A6Sx0lBP8PI

Specs

macOS: 12.6 MacBook Pro M1

nilsreichardt commented 1 year ago

https://user-images.githubusercontent.com/24459435/201410503-9c8fe320-5627-4aae-9fa5-4c040ba694a9.mov

I'm also unable to use the Ubunto (22.04) screen reader. It's working on the Firefox website (as shown in video) but on the Flutter gallery website.

nilsreichardt commented 1 year ago

Same for iOS on Flutter web. Twitter is working, but https://gallery.flutter.dev not.

https://user-images.githubusercontent.com/24459435/201469621-3cb647fa-0664-40d8-96e4-d1f2e64f3354.MP4

nilsreichardt commented 1 year ago

Same for Android on Flutter web. Twitter is working, but https://gallery.flutter.dev/ not.

https://user-images.githubusercontent.com/24459435/201470096-404794aa-2361-4bba-a9a2-116a183c1cfb.mp4

(Somehow the audio is not working. However, you can see in the Video that on the Flutter Gallery page there is never a green bordered box which indicates that this is the current focus of the screen reader).

huycozy commented 1 year ago

Hi @nilsreichardt, thanks for filing the issue. I checked this issue with gallery sample code, I see differences between the two platforms, the behavior is also different from widgets such as Buttons, TextFields:

With macOS Flutter app: The VoiceOver doesn't work when hovering/clicking on Buttons, but works with pressing Tab key on TextFields. Also, instead of pressing Tab key, please try using this key combination: Control + Option + Command + Up/Down.
With Web Flutter app (desktop): The VoiceOver works when using mouse (hovering) on Buttons and TextFields, but using Tab key doesn't work on Buttons.

Demo video (macos and web)

-

flutter doctor -v (stable and master)

```bash [✓] Flutter (Channel stable, 3.3.8, on macOS 13.0 22A380 darwin-x64, locale en-VN) • Flutter version 3.3.8 on channel stable at /Users/huynq/Documents/GitHub/flutter • Upstream repository https://github.com/flutter/flutter.git • Framework revision 52b3dc25f6 (23 hours ago), 2022-11-09 12:09:26 +0800 • Engine revision 857bd6b74c • Dart version 2.18.4 • DevTools version 2.15.0 [✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0) • Android SDK at /Users/huynq/Library/Android/sdk • Platform android-33, build-tools 31.0.0 • ANDROID_HOME = /Users/huynq/Library/Android/sdk • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java • Java version OpenJDK Runtime Environment (build 11.0.13+0-b1751.21-8125866) • All Android licenses accepted. [✓] Xcode - develop for iOS and macOS (Xcode 14.0.1) • Xcode at /Applications/Xcode.app/Contents/Developer • Build 14A400 • CocoaPods version 1.11.3 [✓] Chrome - develop for the web • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome [✓] Android Studio (version 2021.3) • Android Studio at /Applications/Android Studio.app/Contents • Flutter plugin can be installed from: 🔨 https://plugins.jetbrains.com/plugin/9212-flutter • Dart plugin can be installed from: 🔨 https://plugins.jetbrains.com/plugin/6351-dart • Java version OpenJDK Runtime Environment (build 11.0.13+0-b1751.21-8125866) [✓] IntelliJ IDEA Community Edition (version 2022.2.2) • IntelliJ at /Applications/IntelliJ IDEA CE.app • Flutter plugin version 70.0.5 • Dart plugin version 222.4167.21 [✓] IntelliJ IDEA Community Edition (version 2022.1.1) • IntelliJ at /Users/huynq/Library/Application Support/JetBrains/Toolbox/apps/IDEA-C/ch-0/221.5591.52/IntelliJ IDEA CE.app • Flutter plugin can be installed from: 🔨 https://plugins.jetbrains.com/plugin/9212-flutter • Dart plugin can be installed from: 🔨 https://plugins.jetbrains.com/plugin/6351-dart [✓] VS Code (version 1.73.0) • VS Code at /Applications/Visual Studio Code.app/Contents • Flutter extension version 3.52.0 [✓] Connected device (3 available) • iPhone 14 Pro Max (mobile) • 332079AF-E895-4685-910E-7B1E18B0C6B8 • ios • com.apple.CoreSimulator.SimRuntime.iOS-16-0 (simulator) • macOS (desktop) • macos • darwin-x64 • macOS 13.0 22A380 darwin-x64 • Chrome (web) • chrome • web-javascript • Google Chrome 107.0.5304.110 [✓] HTTP Host Availability • All required HTTP hosts are available • No issues found! ``` ```bash [!] Flutter (Channel master, 3.6.0-3.0.pre.1, on macOS 13.0 22A380 darwin-x64, locale en-VN) • Flutter version 3.6.0-3.0.pre.1 on channel master at /Users/huynq/Documents/GitHub/flutter_master • Upstream repository https://github.com/flutter/flutter.git • Framework revision cb224d10e3 (26 minutes ago), 2022-11-13 21:57:38 -0500 • Engine revision b7567b92c7 • Dart version 2.19.0 (build 2.19.0-394.0.dev) • DevTools version 2.19.0 • If those were intentional, you can disregard the above warnings; however it is recommended to use "git" directly to perform update checks and upgrades. [✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0) • Android SDK at /Users/huynq/Library/Android/sdk • Platform android-33, build-tools 31.0.0 • ANDROID_HOME = /Users/huynq/Library/Android/sdk • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java • Java version OpenJDK Runtime Environment (build 11.0.13+0-b1751.21-8125866) • All Android licenses accepted. [✓] Xcode - develop for iOS and macOS (Xcode 14.0.1) • Xcode at /Applications/Xcode.app/Contents/Developer • Build 14A400 • CocoaPods version 1.11.3 [✓] Chrome - develop for the web • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome [✓] Android Studio (version 2021.3) • Android Studio at /Applications/Android Studio.app/Contents • Flutter plugin can be installed from: 🔨 https://plugins.jetbrains.com/plugin/9212-flutter • Dart plugin can be installed from: 🔨 https://plugins.jetbrains.com/plugin/6351-dart • Java version OpenJDK Runtime Environment (build 11.0.13+0-b1751.21-8125866) [✓] IntelliJ IDEA Community Edition (version 2022.2.2) • IntelliJ at /Applications/IntelliJ IDEA CE.app • Flutter plugin version 70.0.5 • Dart plugin version 222.4167.21 [✓] IntelliJ IDEA Community Edition (version 2022.1.1) • IntelliJ at /Users/huynq/Library/Application Support/JetBrains/Toolbox/apps/IDEA-C/ch-0/221.5591.52/IntelliJ IDEA CE.app • Flutter plugin can be installed from: 🔨 https://plugins.jetbrains.com/plugin/9212-flutter • Dart plugin can be installed from: 🔨 https://plugins.jetbrains.com/plugin/6351-dart [✓] VS Code (version 1.73.0) • VS Code at /Applications/Visual Studio Code.app/Contents • Flutter extension version 3.52.0 [✓] Connected device (2 available) • macOS (desktop) • macos • darwin-x64 • macOS 13.0 22A380 darwin-x64 • Chrome (web) • chrome • web-javascript • Google Chrome 107.0.5304.110 [✓] HTTP Host Availability • All required HTTP hosts are available ! Doctor found issues in 1 category. ```

Can you try again with mouse hovering/tab key/key combination to see if it's the same behavior? Maybe the issue is only caused by implementation in the Gallery source code or individual widgets.

nilsreichardt commented 1 year ago

Thanks for taking a look.

However, you can see in your recording (especially in web recording) that the screen reader does not read the content of the website (only when hovering with the mouse).

It should also read the things when using the TAB keys, like shown in this video by the Flutter team: https://youtu.be/A6Sx0lBP8PI. the author of the video says at the beginning that he uses the TAB key to navigate. However, using the TAB key does not result into the same behavior as shown in the video.

Also when we are putting the YouTube aside: How should a blind user navigate through the the Gallery app? The Gallery has accessibility support in the iOS and Android app. But when using the web, you can see the same behavior.

What you can also do: Compare this with non Flutter website / macOS apps. You will see that the TAB key is triggering the screen reader and a blind user would be able to use the app.

huycozy commented 1 year ago

@nilsreichardt Thanks for more information. Labeling this issue for further insights from the team.

yjbanov commented 1 year ago

Our expectation is that when VoiceOver is on the user would use the VoiceOver shortcuts to navigate the UI. By default there the semantics tree is disabled, but the user has the option to click on an accessibility placeholder to enable it. You can also enable it programmatically like this:

void main() {
  runApp(MyApp());
  RendererBinding.instance.setSemanticsEnabled(true);
}

However, it seems in some video recordings you are having trouble finding the placeholder. We should investigate, so I'll keep this issue open.

nilsreichardt commented 4 months ago

Thanks, @yjbanov. For everyone coming to this ticket in 2024 or later, you need to use the following code snippet:

void main() {
  runApp(MyApp());
  if(kIsWeb) {
    // Required on web/desktop to automatically enable accessibility features.
    WidgetsFlutterBinding.ensureInitialized().ensureSemantics();
  }
}

The RendererBinding class doesn't exist anymore.

flutter / flutter