search

flutter / flutter

Flutter makes it easy and fast to build beautiful apps for mobile and beyond
https://flutter.dev
BSD 3-Clause "New" or "Revised" License
170.19k stars 28.5k forks source link

Linux_pixel_7pro integration_ui_keyboard_resize is 2.17% flaky #161300

Closed fluttergithubbot closed 1 week ago

fluttergithubbot commented 4 months ago

The post-submit test builder Linux_pixel_7pro integration_ui_keyboard_resize had a flaky ratio 2.17% for the past (up to) 100 commits, which is above our 2.00% threshold.

One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_pixel_7pro%20integration_ui_keyboard_resize/4489 Commit: https://github.com/flutter/flutter/commit/62c6859e593ba7c7b075ee850c3690eb44401afa

Flaky builds: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_pixel_7pro%20integration_ui_keyboard_resize/4489 https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_pixel_7pro%20integration_ui_keyboard_resize/4470

Recent test runs: https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux_pixel_7pro%20integration_ui_keyboard_resize

Please follow https://github.com/flutter/flutter/blob/master/docs/infra/Reducing-Test-Flakiness.md#fixing-flaky-tests to fix the flakiness and enable the test back after validating the fix (internal dashboard to validate: go/flutter_test_flakiness).

fluttergithubbot commented 3 months ago

[prod pool] flaky ratio for the past (up to) 100 commits between 2025-01-07 and 2025-01-14 is 1.04%. Flaky number: 1; total number: 96. One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_pixel_7pro%20integration_ui_keyboard_resize/4555 Commit: https://github.com/flutter/flutter/commit/e7e5b1c87a87017bc3480bac439867a2b1ed478b Flaky builds: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_pixel_7pro%20integration_ui_keyboard_resize/4555

Recent test runs: https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux_pixel_7pro%20integration_ui_keyboard_resize

fluttergithubbot commented 3 months ago

[prod pool] flaky ratio for the past (up to) 100 commits between 2025-01-11 and 2025-01-21 is 2.11%. Flaky number: 2; total number: 95. One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_pixel_7pro%20integration_ui_keyboard_resize/4712 Commit: https://github.com/flutter/flutter/commit/883ef854134c55470cb437a7cb93749cf8825f5d Flaky builds: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_pixel_7pro%20integration_ui_keyboard_resize/4712 https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_pixel_7pro%20integration_ui_keyboard_resize/4653

Recent test runs: https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux_pixel_7pro%20integration_ui_keyboard_resize

ergosarapu commented 3 months ago

I went through the most recent logs.

Before the test passes successfully there is a line in logs: D/InputMethodManager(23586): showSoftInput() view=io.flutter.embedding.android.FlutterView{c9cf529 VFE...... .F....ID 0,0-1080,2277 #1 aid=1073741824} flags=0 reason=SHOW_SOFT_INPUT

Before the test fails there is a line in logs:

4712 W/InputMethodManager(20089): Ignoring showSoftInput() as view=io.flutter.embedding.android.FlutterView{636da98 VFE...... .F....ID 0,0-1080,2277 #1} is not served.

4653 W/InputMethodManager(24768): Ignoring showSoftInput() as view=io.flutter.embedding.android.FlutterView{1ece7c1 VFE...... .F....ID 0,0-1080,2277 #1} is not served.

4555 W/InputMethodManager(16469): Ignoring showSoftInput() as view=io.flutter.embedding.android.FlutterView{ffbae77 VFE...... .F....ID 0,0-1080,2277 #1} is not served.

4489 W/InputMethodManager(24981): Ignoring showSoftInput() as view=io.flutter.embedding.android.FlutterView{c809714 VFE...... .F....ID 0,0-1080,2277 #1} is not served.

4470 W/InputMethodManager(22919): Ignoring showSoftInput() as view=io.flutter.embedding.android.FlutterView{7d1532d VFE...... .F....ID 0,0-1080,2277 #1} is not served.

4382 This build output had some extra lines in the logs, not sure if they are important

I/ImeTracker(13119): com.yourcompany.integration_ui:93cd2f82: onRequestShow at ORIGIN_CLIENT_SHOW_SOFT_INPUT reason SHOW_SOFT_INPUT
I/ImeTracker(13119): com.yourcompany.integration_ui:93cd2f82: onFailed at PHASE_CLIENT_VIEW_SERVED
W/InputMethodManager(13119): Ignoring showSoftInput() as view=io.flutter.embedding.android.FlutterView{5d7c98e VFE...... .F....ID 0,0-1080,2277 #1} is not served.
harri35 commented 3 months ago

As far as I understand, this part

W/InputMethodManager(20089): Ignoring showSoftInput() as view=io.flutter.embedding.android.FlutterView{636da98 VFE...... .F....ID 0,0-1080,2277 #1} is not served.

says that InputMethodManager ignored the command to show the software keyboard. Because, for some reason, this specific view is deemed to not be in a ready state to receive focus and/or text input. It does not directly specify why.

For the flags, they come from Android's View.toString() - https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/core/java/android/view/View.java;l=6953;drc=f80e786d308318894be30d54b93f38034496fc66

So this specific view is:

Not fully sure about the last few PFLAG_ ones without reading the source more, but as they are same in both success and fail cases then they do not seem relevant atm.

In my previous engineering experience, I have seen the software keyboard fail to open because:

  1. There is a race condition between code calling the keyboard to open and the view being ready. This usually happens if the keyboard open code runs right after the activity / fragments starts and it executes before the view is "ready"
  2. There is something wrong with the operation system, and a restart takes care of it.
ergosarapu commented 3 months ago

Maybe it makes sense to migrate the test to using integration_test instead of flutter_driver directly. Could it make any difference, @andrewkolos?

harri35 commented 3 months ago

I tried to reproduce it locally, seems successfully. I used:

During 113 iterations hyperfine managed to do, the test failed 24 times (and was marked as flaky for it). The flaky test automatic retry saved it until the 113th iteration, where the test failed, and all two retries also failed. See the last log section below. I have the full log also (240MB), I can link it if needed.

[2025-01-23 17:13:47.945025] [STDOUT] stdout: [  +52 ms] executing: /Users/harrikirik/gitrepo/flutter/bin/cache/dart-sdk/bin/dart /Users/harrikirik/gitrepo/flutter/dev/integration_tests/ui/test_driver/keyboard_resize_test.dart
[2025-01-23 17:13:47.964230] [STDOUT] stdout: [  +19 ms] I/WindowExtensionsImpl(14928): Initializing Window Extensions, vendor API level=7, activity embedding enabled=true
[2025-01-23 17:13:47.964250] [STDOUT] stdout: [        ] D/WindowLayoutComponentImpl(14928): Register WindowLayoutInfoListener on Context=com.yourcompany.integration_ui.MainActivity@bea5e74, of which baseContext=android.app.ContextImpl@e00dc40
[2025-01-23 17:13:48.041755] [STDOUT] stdout: [  +77 ms] I/.integration_ui(14928): Compiler allocated 5281KB to compile void android.view.ViewRootImpl.performTraversals()
[2025-01-23 17:13:48.936705] [STDOUT] stdout: [ +894 ms] 00:00 +0: end-to-end test (setUpAll)
[2025-01-23 17:13:48.946538] [STDOUT] stderr: [   +9 ms] VMServiceFlutterDriver: Connecting to Flutter application at http://127.0.0.1:53155/XroU-wv2DlA=/
[2025-01-23 17:13:49.036216] [STDOUT] stderr: [  +90 ms] VMServiceFlutterDriver: Isolate found with number: 6641884694627283
[2025-01-23 17:13:49.058327] [STDOUT] stderr: [  +22 ms] VMServiceFlutterDriver: Isolate 6641884694627283 is runnable.
[2025-01-23 17:13:49.058922] [STDOUT] stderr: [        ] VMServiceFlutterDriver: Isolate is paused at start.
[2025-01-23 17:13:49.058956] [STDOUT] stderr: [        ] VMServiceFlutterDriver: Attempting to resume isolate
[2025-01-23 17:13:49.648152] [STDOUT] stdout: [ +588 ms] I/Choreographer(14928): Skipped 69 frames!  The application may be doing too much work on its main thread.
[2025-01-23 17:13:49.650750] [STDOUT] stderr: [   +2 ms] VMServiceFlutterDriver: Connected to Flutter application.
[2025-01-23 17:13:49.653265] [STDOUT] stdout: [   +2 ms] 00:00 +0: end-to-end test Ensure keyboard dismissal resizes the view to original size
[2025-01-23 17:13:49.826246] [STDOUT] stdout: [ +172 ms] I/ImeTracker(14928): com.yourcompany.integration_ui:fb8b8ee4: onRequestShow at ORIGIN_CLIENT reason SHOW_SOFT_INPUT fromUser false
[2025-01-23 17:13:49.826559] [STDOUT] stdout: [        ] I/ImeTracker(14928): com.yourcompany.integration_ui:fb8b8ee4: onFailed at PHASE_CLIENT_VIEW_SERVED
[2025-01-23 17:13:49.826604] [STDOUT] stdout: [        ] W/InputMethodManager(14928): Ignoring showSoftInput() as view=io.flutter.embedding.android.FlutterView{1d4016e VFE...... .F....ID 0,0-1008,2244 #1} is not served.
[2025-01-23 17:13:49.875977] [STDOUT] stdout: [  +49 ms] I/AssistStructure(14928): Flattened final assist data: 440 bytes, containing 1 windows, 3 views
[2025-01-23 17:13:53.518116] [STDOUT] stdout: [+3642 ms] D/ProfileInstaller(14928): Installing profile for com.yourcompany.integration_ui
[2025-01-23 17:13:56.027086] [STDOUT] stdout: [+2509 ms] 00:07 +0 -1: end-to-end test Ensure keyboard dismissal resizes the view to original size [E]
[2025-01-23 17:13:56.027364] [STDOUT] stdout: [        ]   Expected: true
[2025-01-23 17:13:56.027417] [STDOUT] stdout: [        ]     Actual: <false>
[2025-01-23 17:13:56.032676] [STDOUT] stdout: [   +5 ms]   package:matcher/src/expect/expect.dart 149:31     fail
[2025-01-23 17:13:56.032690] [STDOUT] stdout: [        ]   package:matcher/src/expect/expect.dart 144:3      _expect
[2025-01-23 17:13:56.032694] [STDOUT] stdout: [        ]   package:matcher/src/expect/expect.dart 56:3       expect
[2025-01-23 17:13:56.032697] [STDOUT] stdout: [        ]   test_driver/keyboard_resize_test.dart 44:7        main.<fn>.<fn>
[2025-01-23 17:13:56.032701] [STDOUT] stdout: [        ]   ===== asynchronous gap ===========================
[2025-01-23 17:13:56.032769] [STDOUT] stdout: [        ]   package:test_api/src/backend/declarer.dart 229:9  Declarer.test.<fn>.<fn>
[2025-01-23 17:13:56.032779] [STDOUT] stdout: [        ]   ===== asynchronous gap ===========================
[2025-01-23 17:13:56.032821] [STDOUT] stdout: [        ]   package:test_api/src/backend/declarer.dart 227:7  Declarer.test.<fn>
[2025-01-23 17:13:56.032826] [STDOUT] stdout: [        ]   ===== asynchronous gap ===========================
[2025-01-23 17:13:56.032880] [STDOUT] stdout: [        ]   package:test_api/src/backend/invoker.dart 258:9   Invoker._waitForOutstandingCallbacks.<fn>
[2025-01-23 17:13:56.032936] [STDOUT] stdout: [        ] 00:07 +0 -1: end-to-end test (tearDownAll)
[2025-01-23 17:13:56.038490] [STDOUT] stdout: [   +5 ms] 00:07 +0 -1: Some tests failed.
[2025-01-23 17:13:56.038500] [STDOUT] stdout: [        ] Consider enabling the flag chain-stack-traces to receive more detailed exceptions.
[2025-01-23 17:13:56.038519] [STDOUT] stdout: [        ] For example, 'dart test --chain-stack-traces'.
[2025-01-23 17:13:56.038896] [STDOUT] stderr: [        ] Unhandled exception:
[2025-01-23 17:13:56.038913] [STDOUT] stderr: [        ] Dummy exception to set exit code.
[2025-01-23 17:13:56.049628] [STDOUT] stdout: [  +10 ms] executing: /Users/harrikirik/Library/Android/sdk/platform-tools/adb -s 39221FDJG004TZ shell am force-stop com.yourcompany.integration_ui
[2025-01-23 17:13:56.164939] [STDOUT] stdout: [ +115 ms] executing: /Users/harrikirik/Library/Android/sdk/platform-tools/adb -s 39221FDJG004TZ uninstall com.yourcompany.integration_ui
[2025-01-23 17:13:56.356928] [STDOUT] stdout: [ +191 ms] Success
[2025-01-23 17:13:56.360961] [STDOUT] stdout: [   +4 ms] "flutter drive" took 22,926ms.
[2025-01-23 17:13:56.362535] [STDOUT] stderr: [   +1 ms] 
[2025-01-23 17:13:56.362543] [STDOUT] stderr:            #0      throwToolExit (package:flutter_tools/src/base/common.dart:34:3)
[2025-01-23 17:13:56.362556] [STDOUT] stderr:            #1      DriveCommand.runCommand (package:flutter_tools/src/commands/drive.dart:386:9)
[2025-01-23 17:13:56.362559] [STDOUT] stderr:            <asynchronous suspension>
[2025-01-23 17:13:56.362562] [STDOUT] stderr:            #2      FlutterCommand.run.<anonymous closure> (package:flutter_tools/src/runner/flutter_command.dart:1558:27)
[2025-01-23 17:13:56.362569] [STDOUT] stderr:            <asynchronous suspension>
[2025-01-23 17:13:56.362572] [STDOUT] stderr:            #3      AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:154:19)
[2025-01-23 17:13:56.362582] [STDOUT] stderr:            <asynchronous suspension>
[2025-01-23 17:13:56.362585] [STDOUT] stderr:            #4      CommandRunner.runCommand (package:args/command_runner.dart:212:13)
[2025-01-23 17:13:56.362590] [STDOUT] stderr:            <asynchronous suspension>
[2025-01-23 17:13:56.362668] [STDOUT] stderr:            #5      FlutterCommandRunner.runCommand.<anonymous closure> (package:flutter_tools/src/runner/flutter_command_runner.dart:496:9)
[2025-01-23 17:13:56.362690] [STDOUT] stderr:            <asynchronous suspension>
[2025-01-23 17:13:56.362694] [STDOUT] stderr:            #6      AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:154:19)
[2025-01-23 17:13:56.362698] [STDOUT] stderr:            <asynchronous suspension>
[2025-01-23 17:13:56.362701] [STDOUT] stderr:            #7      FlutterCommandRunner.runCommand (package:flutter_tools/src/runner/flutter_command_runner.dart:431:5)
[2025-01-23 17:13:56.362705] [STDOUT] stderr:            <asynchronous suspension>
[2025-01-23 17:13:56.362707] [STDOUT] stderr:            #8      run.<anonymous closure>.<anonymous closure> (package:flutter_tools/runner.dart:98:11)
[2025-01-23 17:13:56.362710] [STDOUT] stderr:            <asynchronous suspension>
[2025-01-23 17:13:56.362713] [STDOUT] stderr:            #9      AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:154:19)
[2025-01-23 17:13:56.362719] [STDOUT] stderr:            <asynchronous suspension>
[2025-01-23 17:13:56.362729] [STDOUT] stderr:            #10     main (package:flutter_tools/executable.dart:99:3)
[2025-01-23 17:13:56.362732] [STDOUT] stderr:            <asynchronous suspension>
[2025-01-23 17:13:56.362734] [STDOUT] stderr: 
[2025-01-23 17:13:56.362736] [STDOUT] stderr: 
[2025-01-23 17:13:56.362738] [STDOUT] stdout: [        ] Running 3 shutdown hooks
[2025-01-23 17:13:56.363392] [STDOUT] stdout: [        ] Shutdown hooks complete
[2025-01-23 17:13:56.363785] [STDOUT] stdout: [        ] exiting with code 1
[2025-01-23 17:13:56.371301] [STDERR] Task failed: Executable "/Users/harrikirik/gitrepo/flutter/bin/flutter" failed with exit code 1.
[2025-01-23 17:13:56.371415] [STDERR] 
[2025-01-23 17:13:56.371424] [STDERR] Stack trace:
[2025-01-23 17:13:56.379035] [STDERR] package:flutter_devicelab/framework/utils.dart 109:3       fail
[2025-01-23 17:13:56.379043] [STDERR] package:flutter_devicelab/framework/utils.dart 375:5       _execute
[2025-01-23 17:13:56.379047] [STDERR] ===== asynchronous gap ===========================
[2025-01-23 17:13:56.379050] [STDERR] dart:async                                                 _CustomZone.registerBinaryCallback
[2025-01-23 17:13:56.379053] [STDERR] package:flutter_devicelab/framework/framework.dart 309:28  _TaskRunner._performTask.<fn>
[2025-01-23 17:13:56.379056] [STDERR] package:stack_trace                                        Chain.capture
[2025-01-23 17:13:56.379059] [STDERR] package:flutter_devicelab/framework/framework.dart 307:11  _TaskRunner._performTask
[2025-01-23 17:13:56.379062] [STDERR] package:flutter_devicelab/framework/framework.dart 200:43  _TaskRunner.run
[2025-01-23 17:13:56.379065] [STDERR] 
[integration_ui_keyboard_resize] Process terminated with exit code 0.
Task result:
{
  "success": false,
  "reason": "Task failed: Executable \"/Users/harrikirik/gitrepo/flutter/bin/flutter\" failed with exit code 1."
}

═══════════╡ ••• Finished task "integration_ui_keyboard_resize" ••• ╞═══════════

═════════╡ ••• Flaky status for "integration_ui_keyboard_resize" ••• ╞══════════

Consistently failed across all 3 executions.
flaky: false

Looking at the full log - the first test failure came in a matter of minutes. One idea we can try is to turn on the device screen recording and run with zero authorities. Then maybe the video will show us something interesting at the time of failure.

harri35 commented 3 months ago

@andrewkolos CC: @ergosarapu @johnmccutchan

TL;DR Investigated it further. I can reproduce it when running it on Mac, but not when running on Windows (both with Pixel 8 Pro Android 16). I found a possible fix and created a PR for discussion: #162308

Details

My theory is that the failure comes because the await driver.tap(defaultTextField); does not create focus on the defaultTextField in a way that would trigger the keyboard to open.

https://github.com/flutter/flutter/blob/9e273d5e6e2e75336f26951a09aba0f5656a7e77/dev/integration_tests/ui/test_driver/keyboard_resize_test.dart#L29-L35

Instead, the InputMethodManager will ignore the tap and log a warning:

W/InputMethodManager(14928): Ignoring showSoftInput() as view=io.flutter.embedding.android.FlutterView{1d4016e VFE...... .F....ID 0,0-1008,2244 #1} is not served.

Seems to me that the await driver.waitFor(defaultTextField); before the tap is not enough to always ensure the defaultTextField is actually ready on a level that InputMethodManager can focus on and open a keyboard. But it works in most cases because the test runs slowly enough, so it is almost always ready.

Reading the documentation from https://api.flutter.dev/flutter/flutter_driver/FlutterDriver/waitForTappable.html, it seems the waitForTappable() method is a better option to use in this specific case, as it should wait enough to make sure the target is actually tappable. Then I made the replacement and ran the test for 100 iterations I was unable to get the same failure (that I usually was able to reproduce with <=10 iterations). I'll try to run a lot more iterations overnight if possible.

Thoughts?

harri35 commented 3 months ago

My fix in PR #162308 makes it better but does not fix it entirely. The test still failed after 945 iterations (out of 2000) when running it locally.

Hyperfine log

Command terminated with non-zero exit code 1 in benchmark iteration 945.

Test log

[2025-01-28 19:30:16.072056] [STDOUT] stderr: [   +6 ms] VMServiceFlutterDriver: Connected to Flutter application.
[2025-01-28 19:30:16.075422] [STDOUT] stdout: [   +3 ms] 00:00 [32m+0[0m: end-to-end test Ensure keyboard dismissal resizes the view to original size[0m
[2025-01-28 19:30:16.278322] [STDOUT] stdout: [ +201 ms] I/ImeTracker(27642): com.yourcompany.integration_ui:af61f864: onRequestShow at ORIGIN_CLIENT reason SHOW_SOFT_INPUT fromUser false
[2025-01-28 19:30:16.278543] [STDOUT] stdout: [        ] I/ImeTracker(27642): com.yourcompany.integration_ui:af61f864: onFailed at PHASE_CLIENT_VIEW_SERVED
[2025-01-28 19:30:16.281114] [STDOUT] stdout: [   +3 ms] W/InputMethodManager(27642): Ignoring showSoftInput() as view=io.flutter.embedding.android.FlutterView{16610da VFE...... .F....ID 0,0-2244,1008 #1} is not served.
[2025-01-28 19:30:16.322134] [STDOUT] stdout: [  +40 ms] I/AssistStructure(27642): Flattened final assist data: 440 bytes, containing 1 windows, 3 views
[2025-01-28 19:30:18.767436] [STDOUT] stdout: [+2445 ms] D/ProfileInstaller(27642): Installing profile for com.yourcompany.integration_ui
[2025-01-28 19:30:22.489629] [STDOUT] stdout: [+3722 ms] 00:07 [32m+0[0m[31m -1[0m: end-to-end test Ensure keyboard dismissal resizes the view to original size [1m[31m[E][0m[0m
[2025-01-28 19:30:22.490144] [STDOUT] stdout: [        ]   Expected: true
[2025-01-28 19:30:22.490197] [STDOUT] stdout: [        ]     Actual: <false>

The error comes from https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/core/java/android/view/inputmethod/InputMethodManager.java;drc=8fae81cc40d98877629b28a16fcc72b736423bdb;l=2385

Summary Seems using waitForTappable() is insufficient to ensure the TextField can properly receive focus from Android's InputMethodManager and trigger the keyboard to open. Changing my PR back to a draft, back to the drawing board.

Is there anyone we can ask for advice here?

fluttergithubbot commented 3 months ago

[prod pool] flaky ratio for the past (up to) 100 commits between 2025-01-21 and 2025-01-28 is 1.04%. Flaky number: 1; total number: 96. One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_pixel_7pro%20integration_ui_keyboard_resize/4757 Commit: https://github.com/flutter/flutter/commit/23c7ac18e96770158200a07eb1757b53e7d01a7a Flaky builds: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_pixel_7pro%20integration_ui_keyboard_resize/4757

Recent test runs: https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux_pixel_7pro%20integration_ui_keyboard_resize

harri35 commented 3 months ago

Tried again with a slight change - moved the await driver.tap(defaultTextField); inside the loop (https://github.com/flutter/flutter/pull/162308):

Image

and got a surprising result. Now it failed on the 874th iteration (out of 2000) - and all the six taps with 1-second intervals in the loop did got the same result - InputMethodManager refused to open the keyboard:

[2025-01-29 18:55:05.412093] [STDOUT] stdout: [   +2 ms] 00:00 +0: end-to-end test Ensure keyboard dismissal resizes the view to original size
[2025-01-29 18:55:05.649467] [STDOUT] stdout: [ +236 ms] I/ImeTracker(10317): com.yourcompany.integration_ui:f317f69e: onRequestShow at ORIGIN_CLIENT reason SHOW_SOFT_INPUT fromUser false
[2025-01-29 18:55:05.649552] [STDOUT] stdout: [        ] I/ImeTracker(10317): com.yourcompany.integration_ui:f317f69e: onFailed at PHASE_CLIENT_VIEW_SERVED
[2025-01-29 18:55:05.649574] [STDOUT] stdout: [        ] W/InputMethodManager(10317): Ignoring showSoftInput() as view=io.flutter.embedding.android.FlutterView{3002a01 VFE...... .F....ID 0,0-1008,2244 #1} is not served.
[2025-01-29 18:55:05.662344] [STDOUT] stdout: [  +12 ms] I/AssistStructure(10317): Flattened final assist data: 440 bytes, containing 1 windows, 3 views
[2025-01-29 18:55:06.741386] [STDOUT] stdout: [+1079 ms] I/ImeTracker(10317): com.yourcompany.integration_ui:8346e446: onRequestShow at ORIGIN_CLIENT reason SHOW_SOFT_INPUT fromUser false
[2025-01-29 18:55:06.741470] [STDOUT] stdout: [        ] I/ImeTracker(10317): com.yourcompany.integration_ui:8346e446: onFailed at PHASE_CLIENT_VIEW_SERVED
[2025-01-29 18:55:06.746640] [STDOUT] stdout: [   +5 ms] W/InputMethodManager(10317): Ignoring showSoftInput() as view=io.flutter.embedding.android.FlutterView{3002a01 VFE...... .F...... 0,0-1008,2244 #1 aid=1073741824} is not served.
[2025-01-29 18:55:07.839947] [STDOUT] stdout: [+1093 ms] I/ImeTracker(10317): com.yourcompany.integration_ui:4a084cea: onRequestShow at ORIGIN_CLIENT reason SHOW_SOFT_INPUT fromUser false
[2025-01-29 18:55:07.840042] [STDOUT] stdout: [        ] I/ImeTracker(10317): com.yourcompany.integration_ui:4a084cea: onFailed at PHASE_CLIENT_VIEW_SERVED
[2025-01-29 18:55:07.840668] [STDOUT] stdout: [        ] W/InputMethodManager(10317): Ignoring showSoftInput() as view=io.flutter.embedding.android.FlutterView{3002a01 VFE...... .F...... 0,0-1008,2244 #1 aid=1073741824} is not served.
[2025-01-29 18:55:08.727377] [STDOUT] stdout: [ +886 ms] D/ProfileInstaller(10317): Installing profile for com.yourcompany.integration_ui
[2025-01-29 18:55:08.953651] [STDOUT] stdout: [ +226 ms] I/ImeTracker(10317): com.yourcompany.integration_ui:98fcfae1: onRequestShow at ORIGIN_CLIENT reason SHOW_SOFT_INPUT fromUser false
[2025-01-29 18:55:08.953731] [STDOUT] stdout: [        ] I/ImeTracker(10317): com.yourcompany.integration_ui:98fcfae1: onFailed at PHASE_CLIENT_VIEW_SERVED
[2025-01-29 18:55:08.955051] [STDOUT] stdout: [   +1 ms] W/InputMethodManager(10317): Ignoring showSoftInput() as view=io.flutter.embedding.android.FlutterView{3002a01 VFE...... .F...... 0,0-1008,2244 #1 aid=1073741824} is not served.
[2025-01-29 18:55:10.050636] [STDOUT] stdout: [+1095 ms] I/ImeTracker(10317): com.yourcompany.integration_ui:753b85dd: onRequestShow at ORIGIN_CLIENT reason SHOW_SOFT_INPUT fromUser false
[2025-01-29 18:55:10.050769] [STDOUT] stdout: [        ] I/ImeTracker(10317): com.yourcompany.integration_ui:753b85dd: onFailed at PHASE_CLIENT_VIEW_SERVED
[2025-01-29 18:55:10.051398] [STDOUT] stdout: [        ] W/InputMethodManager(10317): Ignoring showSoftInput() as view=io.flutter.embedding.android.FlutterView{3002a01 VFE...... .F...... 0,0-1008,2244 #1 aid=1073741824} is not served.
[2025-01-29 18:55:11.203107] [STDOUT] stdout: [+1151 ms] I/ImeTracker(10317): com.yourcompany.integration_ui:8e37f6cc: onRequestShow at ORIGIN_CLIENT reason SHOW_SOFT_INPUT fromUser false
[2025-01-29 18:55:11.203192] [STDOUT] stdout: [        ] I/ImeTracker(10317): com.yourcompany.integration_ui:8e37f6cc: onFailed at PHASE_CLIENT_VIEW_SERVED
[2025-01-29 18:55:11.203217] [STDOUT] stdout: [        ] W/InputMethodManager(10317): Ignoring showSoftInput() as view=io.flutter.embedding.android.FlutterView{3002a01 VFE...... .F...... 0,0-1008,2244 #1 aid=1073741824} is not served.
[2025-01-29 18:55:12.225410] [STDOUT] stdout: [+1022 ms] 00:07 +0 -1: end-to-end test Ensure keyboard dismissal resizes the view to original size [E]

full log of the test run here: https://drive.google.com/file/d/1C-JJejQUmesoKSL4xJEhiwd1XbkF0YRj/view?usp=sharing

@reidbaker When you have some time, could you please take a quick look? Is this here an actual bug (on Android, Flutter or Flutter Driver side)? All the previous tests and context is in this thread in my previous comments.

andrewkolos commented 3 months ago

I'm sorry that I haven't found the time to dig deeper here. For what it's worth, this doesn't appear to be the first time this test has run this issue: https://github.com/flutter/flutter/pull/141301

harri35 commented 3 months ago

Quick update - As this is locally reproducible for me, I have tried the following things: 1) Using waitForTappable() instead of waitFor() - No improvement 2) Moving the await driver.tap(defaultTextField); inside the loop (https://github.com/flutter/flutter/issues/161300#issuecomment-2623652160) - No improvement (all 6 attempts to open failed) 3) Change in 2 + calling the hide keyboard logic after opening the keyboard fails and retrying to open the keyboard after that (show-fail-hide-show) - No improvement (all 6+6 attempts to open failed).

This looks strange. I am starting to think that the TextField or Android's InputMethodManager or some underlying thing is in an unexpected state here in some rare cases. Eg that just "waiting a longer" doesn't help

fluttergithubbot commented 3 months ago

[prod pool] flaky ratio for the past (up to) 100 commits between 2025-01-29 and 2025-02-04 is 0.00%. Flaky number: 0; total number: 85.

harri35 commented 3 months ago

Quick update - additional things I tried when locally running the test:

  1. If the keyboard is not opening, then calling textField.focusNode!.unfocus(); and trying to open the keyboard again - No improvement (failed at 343/2000)
  2. If the keyboard is not opening then trying to click on to a different TextField with a separate FocusNone - No improvement (failed at 289/2000)

Error is still the same:

I/ImeTracker( 7948): com.yourcompany.integration_ui:c7aa4db5: onRequestShow at ORIGIN_CLIENT reason SHOW_SOFT_INPUT fromUser false
I/ImeTracker( 7948): com.yourcompany.integration_ui:c7aa4db5: onFailed at PHASE_CLIENT_VIEW_SERVED
W/InputMethodManager( 7948): Ignoring showSoftInput() as view=io.flutter.embedding.android.FlutterView{36b0473 VFE...... .F...... 0,0-1008,2244 #1 aid=1073741824} is not served.

Is this an actual bug and something broken or in an invalid state between the InputMethodManager and FlutterView?

Please note that the tweaks I made in https://github.com/flutter/flutter/pull/162308 still make the test a lot less flaky locally, just they do not fix it fully. I propose to 1) Merge my PR https://github.com/flutter/flutter/pull/162308 as it makes the test faster and less flaky. I'll add the reasoning there 2) Create a bug task for this issue for the Android team

fluttergithubbot commented 2 months ago

[prod pool] flaky ratio for the past (up to) 100 commits between 2025-02-04 and 2025-02-11 is 4.30%. Flaky number: 4; total number: 93. One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_pixel_7pro%20integration_ui_keyboard_resize/5063 Commit: https://github.com/flutter/flutter/commit/30b8eb635bd4e85bf70faccdb2f7a23608173aed Flaky builds: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_pixel_7pro%20integration_ui_keyboard_resize/5063 https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_pixel_7pro%20integration_ui_keyboard_resize/5058 https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_pixel_7pro%20integration_ui_keyboard_resize/5039 https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_pixel_7pro%20integration_ui_keyboard_resize/4992

Recent test runs: https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux_pixel_7pro%20integration_ui_keyboard_resize

fluttergithubbot commented 2 months ago

[prod pool] flaky ratio for the past (up to) 100 commits between 2025-02-11 and 2025-02-18 is 1.03%. Flaky number: 1; total number: 97. One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_pixel_7pro%20integration_ui_keyboard_resize/5090 Commit: https://github.com/flutter/flutter/commit/17ea9569da31b2d718fe3f9114d8b1b251b95f8b Flaky builds: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_pixel_7pro%20integration_ui_keyboard_resize/5090

Recent test runs: https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux_pixel_7pro%20integration_ui_keyboard_resize

harri35 commented 2 months ago

Update

fluttergithubbot commented 2 months ago

[prod pool] flaky ratio for the past (up to) 100 commits between 2025-02-19 and 2025-02-25 is 2.30%. Flaky number: 2; total number: 87. One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_pixel_7pro%20integration_ui_keyboard_resize/5280 Commit: https://github.com/flutter/flutter/commit/dec3b1e20e9d4a5477f32b6a2f3a172edaee886d Flaky builds: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_pixel_7pro%20integration_ui_keyboard_resize/5280 https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_pixel_7pro%20integration_ui_keyboard_resize/5228

Recent test runs: https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux_pixel_7pro%20integration_ui_keyboard_resize

fluttergithubbot commented 2 months ago

[prod pool] flaky ratio for the past (up to) 100 commits between 2025-02-27 and 2025-03-04 is 1.04%. Flaky number: 1; total number: 96. One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_pixel_7pro%20integration_ui_keyboard_resize/5454 Commit: https://github.com/flutter/flutter/commit/245abd342aa92a2e2b2b7902ea7fd38b01f36186 Flaky builds: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_pixel_7pro%20integration_ui_keyboard_resize/5454

Recent test runs: https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux_pixel_7pro%20integration_ui_keyboard_resize

harri35 commented 2 months ago

The workaround PR was merged: https://github.com/flutter/flutter/pull/162308 Let's wait for the next stats, and if the numbers improve, then close this.

Note: The issue for fixing the source of the problem is here: https://github.com/flutter/flutter/issues/163606. But if the stats improves, then I would close this without waiting for that one

fluttergithubbot commented 2 months ago

[prod pool] flaky ratio for the past (up to) 100 commits between 2025-03-04 and 2025-03-11 is 0.00%. Flaky number: 0; total number: 99.

harri35 commented 2 months ago

Closing this as a workaround was implemented, and the stats look better.

johnmccutchan commented 1 month ago

Reopening issue as we need to root cause the flake

harri35 commented 1 month ago

Reopening issue as we need to root cause the flake

@johnmccutchan Totally agree, we need to find the real cause. There is a bug report for the FlutterDriver and both reproduction info and a reproduction video from a second source here https://github.com/flutter/flutter/issues/163606 I was thinking the work on the root cause continues there; that is why I initially closed this one.

fluttergithubbot commented 1 month ago

[prod pool] flaky ratio for the past (up to) 100 commits between 2025-03-11 and 2025-03-18 is 0.00%. Flaky number: 0; total number: 88.

Piinks commented 1 month ago

I am going to mark this as a P1 for root cause analysis (per comment) as it appears it is not flaking right now. :)

fluttergithubbot commented 1 month ago

[prod pool] flaky ratio for the past (up to) 100 commits between 2025-03-18 and 2025-03-25 is 1.00%. Flaky number: 1; total number: 100. One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_pixel_7pro%20integration_ui_keyboard_resize/5760 Commit: https://github.com/flutter/flutter/commit/5a8d4032e9460e9cf4fe948cbde7009baba45849 Flaky builds: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_pixel_7pro%20integration_ui_keyboard_resize/5760

Recent test runs: https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux_pixel_7pro%20integration_ui_keyboard_resize

fluttergithubbot commented 1 month ago

[prod pool] flaky ratio for the past (up to) 100 commits between 2025-03-26 and 2025-04-01 is 0.00%. Flaky number: 0; total number: 100.

yjbanov commented 1 month ago

I'm unassigning from myself. The goal of my assignment was to verify the hypothesis that the FlutterDriver.tap method is flaky. I have confirmed that it is not. The flaky part is in the actual functionality of the virtual keyboard that's being tested. FlutterDriver itself does its job admirably.

However, the issue should remain open, because we still should decide what to do about this test. If we have no control over the reliability of the keyboard-screen size interaction, we might want to remove this test as infeasible, or find a different strategy for deflaking it (e.g. use a different retry strategy).

fluttergithubbot commented 1 month ago

[prod pool] flaky ratio for the past (up to) 100 commits between 2025-04-01 and 2025-04-08 is 0.00%. Flaky number: 0; total number: 99.

fluttergithubbot commented 3 weeks ago

[prod pool] flaky ratio for the past (up to) 100 commits between 2025-04-08 and 2025-04-15 is 1.11%. Flaky number: 1; total number: 90. One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_pixel_7pro%20integration_ui_keyboard_resize/6144 Commit: https://github.com/flutter/flutter/commit/dc78960cdbcab5cd3989ffa9bf8a462e187ed2ad Flaky builds: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_pixel_7pro%20integration_ui_keyboard_resize/6144

Recent test runs: https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux_pixel_7pro%20integration_ui_keyboard_resize

fluttergithubbot commented 2 weeks ago

[prod pool] flaky ratio for the past (up to) 100 commits between 2025-04-15 and 2025-04-22 is 1.11%. Flaky number: 1; total number: 90. One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_pixel_7pro%20integration_ui_keyboard_resize/6298 Commit: https://github.com/flutter/flutter/commit/5816a496295cd0f62c5276202cbacb8c9516c3e7 Flaky builds: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_pixel_7pro%20integration_ui_keyboard_resize/6298

Recent test runs: https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux_pixel_7pro%20integration_ui_keyboard_resize

fluttergithubbot commented 1 week ago

[prod pool] flaky ratio for the past (up to) 100 commits between 2025-04-23 and 2025-04-29 is 1.08%. Flaky number: 1; total number: 93. One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_pixel_7pro%20integration_ui_keyboard_resize/6360 Commit: https://github.com/flutter/flutter/commit/83483976ec0c2667bb991d052a0e02c691687bf2 Flaky builds: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux_pixel_7pro%20integration_ui_keyboard_resize/6360

Recent test runs: https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux_pixel_7pro%20integration_ui_keyboard_resize

bkonyi commented 1 week ago

The most recent flakes from the past few weeks are due to DeviceException: No Android devices detected, which is currently being tracked by separate issues (https://github.com/flutter/flutter/issues/167638, https://github.com/flutter/flutter/issues/168055). Since the original flake seems to have been resolved by https://github.com/flutter/flutter/pull/162308, I'm going to close this so new issues will be created for new flakes going forward.