search

flutter / website

Flutter documentation web site
https://docs.flutter.dev
Other
2.8k stars 3.21k forks source link

[Codelab] Example failing for Scrolling test in integration testing. #6508

Closed chonghorizons closed 2 years ago

chonghorizons commented 2 years ago

Expected Result

https://codelabs.developers.google.com/codelabs/flutter-app-testing/ The codelab integration test examples run and the tests pass.

Actual Result

The Scrolling test fails in https://github.com/flutter/codelabs/blob/master/testing_codelab/step_07/integration_test/app_test.dart

The other two tests pass.

This is the error log ``` shell$ flutter drive --driver integration_test/driver.dart --target integration_test/app_test.dart --profile Running "flutter pub get" in testing_app... 1,543ms Running Gradle task 'assembleProfile'... 56.9s ✓ Built build/app/outputs/flutter-apk/app-profile.apk (11.8MB). Installing build/app/outputs/flutter-apk/app.apk... 3.5s VMServiceFlutterDriver: Connecting to Flutter application at http://127.0.0.1:62747/iRiezfewcsE=/ VMServiceFlutterDriver: Isolate found with number: 2816534710512779 VMServiceFlutterDriver: Isolate is paused at start. VMServiceFlutterDriver: Attempting to resume isolate I/flutter ( 5749): 00:00 +0: Testing App Performance Tests Favorites operations test VMServiceFlutterDriver: Connected to Flutter application. I/flutter ( 5749): 00:09 +1: Testing App Performance Tests (tearDownAll) W/ple.testing_ap( 5749): Accessing hidden method Lsun/misc/Unsafe;->compareAndSwapObject(Ljava/lang/Object;JLjava/lang/Object;Ljava/lang/Object;)Z (greylist, linking, allowed) W/ple.testing_ap( 5749): Accessing hidden method Lsun/misc/Unsafe;->compareAndSwapObject(Ljava/lang/Object;JLjava/lang/Object;Ljava/lang/Object;)Z (greylist, linking, allowed) I/flutter ( 5749): 00:09 +2: All tests passed! All tests passed. Macs-MacBook-Air:testing_app howardchong$ flutter drive --driver integration_test/driver.dart --target integration_test/app_test.dart --profile Running "flutter pub get" in testing_app... 1,470ms Running Gradle task 'assembleProfile'... 65.0s ✓ Built build/app/outputs/flutter-apk/app-profile.apk (14.6MB). Installing build/app/outputs/flutter-apk/app.apk... 3.8s VMServiceFlutterDriver: Connecting to Flutter application at http://127.0.0.1:62934/OSWzRBQSSno=/ VMServiceFlutterDriver: Isolate found with number: 2692430830870515 VMServiceFlutterDriver: Isolate is paused at start. VMServiceFlutterDriver: Attempting to resume isolate I/flutter ( 6418): 00:00 +0: Testing App Performance Tests Scrolling test VMServiceFlutterDriver: Connected to Flutter application. I/flutter ( 6418): SocketException: OS Error: Connection refused, errno = 111, address = localhost, port = 40625 I/flutter ( 6418): #0 _NativeSocket.startConnect (dart:io-patch/socket_patch.dart:677) I/flutter ( 6418): #1 _RawSocket.startConnect (dart:io-patch/socket_patch.dart:1680) I/flutter ( 6418): #2 RawSocket.startConnect (dart:io-patch/socket_patch.dart:27) I/flutter ( 6418): #3 Socket._startConnect (dart:io-patch/socket_patch.dart:1891) I/flutter ( 6418): #4 Socket.startConnect (dart:io/socket.dart:738) I/flutter ( 6418): #5 _ConnectionTarget.connect (dart:_http/http_impl.dart:2439) I/flutter ( 6418): #6 _HttpClient._getConnection.connect (dart:_http/http_impl.dart:2805) I/flutter ( 6418): #7 _HttpClient._getConnection (dart:_http/http_impl.dart:2810) I/flutter ( 6418): #8 _HttpClient._openUrl (dart:_http/http_impl.dart:2694) I/flutter ( 6418): #9 _HttpClient.openUrl (dart:_http/http_impl.dart:2568) I/flutter ( 6418): #10 _WebSocketImpl.connect (dart:_http/websocket_impl.dart:1027) I/flutter ( 6418): #11 WebSocket.connect (dart:_http/websocket.dart:379) I/flutter ( 6418): #12 vmServiceConnectUri (package:vm_service/vm_service_io.dart:32) I/flutter ( 6418): #13 IntegrationTestWidgetsFlutterBinding.enableTimeline (package:integration_test/integration_test.dart:244) I/flutter ( 6418): I/flutter ( 6418): #14 IntegrationTestWidgetsFlutterBinding.traceTimeline (package:integration_test/integration_test.dart:269) I/flutter ( 6418): I/flutter ( 6418): #15 IntegrationTestWidgetsFlutterBinding._runAndGetGCInfo (package:integration_test/integration_test.dart:321) I/flutter ( 6418): I/flutter ( 6418): #16 IntegrationTestWidgetsFlutterBinding.watchPerformance (package:integration_test/integration_test.dart:376) I/flutter ( 6418): I/flutter ( 6418): #17 main.. (file:///Users/howardchong/Proj2021/testing_app/integration_test/app_test.dart:22) I/flutter ( 6418): I/flutter ( 6418): #18 testWidgets.. (package:flutter_test/src/widget_tester.dart:176) I/flutter ( 6418): I/flutter ( 6418): #19 TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:817) I/flutter ( 6418): I/flutter ( 6418): 00:02 +0: Testing App Performance Tests Scrolling test [E] I/flutter ( 6418): Test failed. See exception logs above. I/flutter ( 6418): The test description was: Scrolling test I/flutter ( 6418): I/flutter ( 6418): Consider enabling the flag chain-stack-traces to receive more detailed exceptions. I/flutter ( 6418): For example, 'dart test --chain-stack-traces'. I/flutter ( 6418): 00:02 +0 -1: Testing App Performance Tests Favorites operations test I/flutter ( 6418): 00:11 +1 -1: Testing App Performance Tests (tearDownAll) W/ple.testing_ap( 6418): Accessing hidden method Lsun/misc/Unsafe;->compareAndSwapObject(Ljava/lang/Object;JLjava/lang/Object;Ljava/lang/Object;)Z (greylist, linking, allowed) W/ple.testing_ap( 6418): Accessing hidden method Lsun/misc/Unsafe;->compareAndSwapObject(Ljava/lang/Object;JLjava/lang/Object;Ljava/lang/Object;)Z (greylist, linking, allowed) I/flutter ( 6418): 00:11 +2 -1: Some tests failed. Failure Details: Failure in method: Scrolling test ``` end of failure 1
This is flutter doctor ``` flutter doctor -v [✓] Flutter (Channel stable, 2.5.3, on macOS 12.0.1 21A559 darwin-x64, locale en-US) • Flutter version 2.5.3 at /Users/howardchong/development/flutter • Upstream repository https://github.com/flutter/flutter.git • Framework revision 18116933e7 (6 weeks ago), 2021-10-15 10:46:35 -0700 • Engine revision d3ea636dc5 • Dart version 2.14.4 [✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0) • Android SDK at /Users/howardchong/Library/Android/sdk • Platform android-31, build-tools 31.0.0 • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7281165) • All Android licenses accepted. [✗] Xcode - develop for iOS and macOS ✗ Xcode installation is incomplete; a full installation is necessary for iOS development. Download at: https://developer.apple.com/xcode/download/ Or install Xcode via the App Store. Once installed, run: sudo xcode-select --switch /Applications/Xcode.app/Contents/Developer sudo xcodebuild -runFirstLaunch ✗ CocoaPods installed but not working. You appear to have CocoaPods installed but it is not working. This can happen if the version of Ruby that CocoaPods was installed with is different from the one being used to invoke it. This can usually be fixed by re-installing CocoaPods. To re-install see https://guides.cocoapods.org/using/getting-started.html#installation for instructions. [✓] Chrome - develop for the web • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome [✓] Android Studio (version 2020.3) • Android Studio at /Applications/Android Studio.app/Contents • Flutter plugin can be installed from: 🔨 https://plugins.jetbrains.com/plugin/9212-flutter • Dart plugin can be installed from: 🔨 https://plugins.jetbrains.com/plugin/6351-dart • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7281165) [✓] Connected device (2 available) • PH 1 (mobile) • PM1LHMA7B1300032 • android-arm64 • Android 10 (API 29) • Chrome (web) • chrome • web-javascript • Google Chrome 96.0.4664.55 ```

Running on physical device (PH1).

danagbemava-nc commented 2 years ago

@chonghorizons, thanks for reporting this.

I looked into it and the issue appears to be coming from the binding.watchPerformance() method.

logs without `binding.watchPerformance` ```console nexus@Daniels-MBP testing_app % flutter drive --driver integration_test/driver.dart --target integration_test/app_test.dart --profile Running "flutter pub get" in testing_app... 989ms Multiple devices found: M2007J20CG (mobile) • 5dd3be00 • android-arm64 • Android 11 (API 30) sdk gphone arm64 (mobile) • emulator-5554 • android-arm64 • Android 11 (API 30) (emulator) macOS (desktop) • macos • darwin-arm64 • macOS 11.6 20G165 darwin-arm Chrome (web) • chrome • web-javascript • Google Chrome 96.0.4664.55 [1]: M2007J20CG (5dd3be00) [2]: sdk gphone arm64 (emulator-5554) [3]: macOS (macos) [4]: Chrome (chrome) Please choose one (To quit, press "q/Q"): 1 Warning: Mapping new ns http://schemas.android.com/repository/android/common/02 to old ns http://schemas.android.com/repository/android/common/01 Warning: Mapping new ns http://schemas.android.com/repository/android/generic/02 to old ns http://schemas.android.com/repository/android/generic/01 Warning: Mapping new ns http://schemas.android.com/sdk/android/repo/addon2/02 to old ns http://schemas.android.com/sdk/android/repo/addon2/01 Warning: Mapping new ns http://schemas.android.com/sdk/android/repo/repository2/02 to old ns http://schemas.android.com/sdk/android/repo/repository2/01 Warning: Mapping new ns http://schemas.android.com/sdk/android/repo/sys-img2/02 to old ns http://schemas.android.com/sdk/android/repo/sys-img2/01 Running Gradle task 'assembleProfile'... 26.7s ✓ Built build/app/outputs/flutter-apk/app-profile.apk (11.8MB). Installing build/app/outputs/flutter-apk/app.apk... 5.6s VMServiceFlutterDriver: Connecting to Flutter application at http://127.0.0.1:55581/KiNkCzrDoWQ=/ VMServiceFlutterDriver: Isolate found with number: 3842243781551043 VMServiceFlutterDriver: Isolate is paused at start. VMServiceFlutterDriver: Attempting to resume isolate I/flutter (13678): 00:00 +0: Testing App Performance Tests Scrolling test VMServiceFlutterDriver: Connected to Flutter application. I/ple.testing_ap(13678): ProcessProfilingInfo new_methods=538 is saved saved_to_disk=1 resolve_classes_delay=8000 I/flutter (13678): 00:04 +1: Testing App Performance Tests Favorites operations test I/flutter (13678): 00:13 +2: Testing App Performance Tests (tearDownAll) I/flutter (13678): 00:13 +3: All tests passed! All tests passed. ``` with `binding.watchPerformance` ```console nexus@Daniels-MBP testing_app % flutter drive --driver integration_test/driver.dart --target integration_test/app_test.dart --profile Running "flutter pub get" in testing_app... 993ms Multiple devices found: M2007J20CG (mobile) • 5dd3be00 • android-arm64 • Android 11 (API 30) sdk gphone arm64 (mobile) • emulator-5554 • android-arm64 • Android 11 (API 30) (emulator) macOS (desktop) • macos • darwin-arm64 • macOS 11.6 20G165 darwin-arm Chrome (web) • chrome • web-javascript • Google Chrome 96.0.4664.55 [1]: M2007J20CG (5dd3be00) [2]: sdk gphone arm64 (emulator-5554) [3]: macOS (macos) [4]: Chrome (chrome) Please choose one (To quit, press "q/Q"): 1 Warning: Mapping new ns http://schemas.android.com/repository/android/common/02 to old ns http://schemas.android.com/repository/android/common/01 Warning: Mapping new ns http://schemas.android.com/repository/android/generic/02 to old ns http://schemas.android.com/repository/android/generic/01 Warning: Mapping new ns http://schemas.android.com/sdk/android/repo/addon2/02 to old ns http://schemas.android.com/sdk/android/repo/addon2/01 Warning: Mapping new ns http://schemas.android.com/sdk/android/repo/repository2/02 to old ns http://schemas.android.com/sdk/android/repo/repository2/01 Warning: Mapping new ns http://schemas.android.com/sdk/android/repo/sys-img2/02 to old ns http://schemas.android.com/sdk/android/repo/sys-img2/01 Running Gradle task 'assembleProfile'... 28.3s ✓ Built build/app/outputs/flutter-apk/app-profile.apk (14.6MB). Installing build/app/outputs/flutter-apk/app.apk... 6.2s VMServiceFlutterDriver: Connecting to Flutter application at http://127.0.0.1:56222/HgzzoqWRc2o=/ VMServiceFlutterDriver: Isolate found with number: 1412602457582979 VMServiceFlutterDriver: Isolate is paused at start. VMServiceFlutterDriver: Attempting to resume isolate I/flutter (15854): 00:00 +0: Testing App Performance Tests Scrolling test VMServiceFlutterDriver: Connected to Flutter application. I/ple.testing_ap(15854): WaitForGcToComplete blocked RunEmptyCheckpoint on ProfileSaver for 7.963ms I/ple.testing_ap(15854): ProcessProfilingInfo new_methods=542 is saved saved_to_disk=1 resolve_classes_delay=8000 I/flutter (15854): SocketException: OS Error: Connection refused, errno = 111, address = localhost, port = 39192 I/flutter (15854): #0 _NativeSocket.startConnect (dart:io-patch/socket_patch.dart:677) I/flutter (15854): #1 _RawSocket.startConnect (dart:io-patch/socket_patch.dart:1680) I/flutter (15854): #2 RawSocket.startConnect (dart:io-patch/socket_patch.dart:27) I/flutter (15854): #3 Socket._startConnect (dart:io-patch/socket_patch.dart:1891) I/flutter (15854): #4 Socket.startConnect (dart:io/socket.dart:738) I/flutter (15854): #5 _ConnectionTarget.connect (dart:_http/http_impl.dart:2439) I/flutter (15854): #6 _HttpClient._getConnection.connect (dart:_http/http_impl.dart:2805) I/flutter (15854): #7 _HttpClient._getConnection (dart:_http/http_impl.dart:2810) I/flutter (15854): #8 _HttpClient._openUrl (dart:_http/http_impl.dart:2694) I/flutter (15854): #9 _HttpClient.openUrl (dart:_http/http_impl.dart:2568) I/flutter (15854): #10 _WebSocketImpl.connect (dart:_http/websocket_impl.dart:1027) I/flutter (15854): #11 WebSocket.connect (dart:_http/websocket.dart:379) I/flutter (15854): #12 vmServiceConnectUri (package:vm_service/vm_service_io.dart:32) I/flutter (15854): #13 IntegrationTestWidgetsFlutterBinding.enableTimeline (package:integration_test/integration_test.dart:244) I/flutter (15854): I/flutter (15854): #14 IntegrationTestWidgetsFlutterBinding.traceTimeline (package:integration_test/integration_test.dart:269) I/flutter (15854): I/flutter (15854): #15 IntegrationTestWidgetsFlutterBinding._runAndGetGCInfo (package:integration_test/integration_test.dart:321) I/flutter (15854): I/flutter (15854): #16 IntegrationTestWidgetsFlutterBinding.watchPerformance (package:integration_test/integration_test.dart:376) I/flutter (15854): I/flutter (15854): #17 main.. (file:///Users/nexus/projects/nevercode/testing_app/integration_test/app_test.dart:18) I/flutter (15854): I/flutter (15854): #18 testWidgets.. (package:flutter_test/src/widget_tester.dart:176) I/flutter (15854): I/flutter (15854): #19 TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:817) I/flutter (15854): I/flutter (15854): 00:02 +0: Testing App Performance Tests Scrolling test [E] I/flutter (15854): Test failed. See exception logs above. I/flutter (15854): The test description was: Scrolling test I/flutter (15854): I/flutter (15854): Consider enabling the flag chain-stack-traces to receive more detailed exceptions. I/flutter (15854): For example, 'dart test --chain-stack-traces'. I/flutter (15854): 00:02 +0 -1: Testing App Performance Tests Favorites operations test I/flutter (15854): 00:10 +1 -1: Testing App Performance Tests (tearDownAll) I/flutter (15854): 00:10 +2 -1: Some tests failed. Failure Details: Failure in method: Scrolling test end of failure 1 ```

/cc @domesticmouse

chonghorizons commented 2 years ago

Small Update:

I tried to run this on the flutter web, and I got all tests passed. In --no-headless mode, I was able to see the two "fling" actions.

Log for `-d web-server` ```console testing_app howardchong$ flutter drive --driver integration_test/driver.dart --target integration_test/app_test.dart -d web-server --no-headless Running "flutter pub get" in testing_app... 2,383ms Launching integration_test/app_test.dart on Web Server in debug mode... Waiting for connection from debug service on Web Server... 35.0s integration_test/app_test.dart is being served at http://localhost:53303 The web-server device requires the Dart Debug Chrome extension for debugging. Consider using the Chrome or Edge devices for an improved development workflow. [INFO]: http://localhost:53303/ 61:26 "Installed new service worker." [WARNING]: http://localhost:53303/dart_sdk.js 49581:16 "registerExtension() from dart:developer is only supported in build/run/test environments where the developer event method hooks have been set." [WARNING]: http://localhost:53303/dart_sdk.js 49563:16 "postEvent() from dart:developer is only supported in build/run/test environments where the developer event method hooks have been set." [INFO]: http://localhost:53303/dart_sdk.js 28067:14 "00:00 +0: Testing App Performance Tests Scrolling test" All tests passed. Application finished. ```

Therefore this may be a physical device error. I get the binding.watchPerformance() error you get and we both tested on physical devices.

Current knowledge

Suggestions

See if the same error triggers when running in the Android Emulator or an iOS device.

danagbemava-nc commented 2 years ago

It failed on iOS too.

logs ```console Running "flutter pub get" in testing_app... 949ms Automatically signing iOS for device deployment using specified development team in Xcode project: HGS96587A5 Running pod install... 909ms Running Xcode build... └─Compiling, linking and signing... 12.3s Xcode build done. 49.5s Installing and launching... 38.7s VMServiceFlutterDriver: Connecting to Flutter application at http://127.0.0.1:49672/YeaX-qIB1wA=/ VMServiceFlutterDriver: Isolate found with number: 15265507526239 VMServiceFlutterDriver: Isolate is paused at start. VMServiceFlutterDriver: Attempting to resume isolate flutter: 00:00 +0: Testing App Performance Tests Scrolling test fopen failed for data file: errno = 2 (No such file or directory) Errors found! Invalidating cache... fopen failed for data file: errno = 2 (No such file or directory) Errors found! Invalidating cache... VMServiceFlutterDriver: Connected to Flutter application. flutter: SocketException: OS Error: Connection refused, errno = 61, address = localhost, port = 51806 flutter: #0 _NativeSocket.startConnect (dart:io-patch/socket_patch.dart:677) flutter: #1 _RawSocket.startConnect (dart:io-patch/socket_patch.dart:1680) flutter: #2 RawSocket.startConnect (dart:io-patch/socket_patch.dart:27) flutter: #3 Socket._startConnect (dart:io-patch/socket_patch.dart:1891) flutter: #4 Socket.startConnect (dart:io/socket.dart:738) flutter: #5 _ConnectionTarget.connect (dart:_http/http_impl.dart:2439) flutter: #6 _HttpClient._getConnection.connect (dart:_http/http_impl.dart:2805) flutter: #7 _HttpClient._getConnection (dart:_http/http_impl.dart:2810) flutter: #8 _HttpClient._openUrl (dart:_http/http_impl.dart:2694) flutter: #9 _HttpClient.openUrl (dart:_http/http_impl.dart:2568) flutter: #10 _WebSocketImpl.connect (dart:_http/websocket_impl.dart:1027) flutter: #11 WebSocket.connect (dart:_http/websocket.dart:379) flutter: #12 vmServiceConnectUri (package:vm_service/vm_service_io.dart:32) flutter: #13 IntegrationTestWidgetsFlutterBinding.enableTimeline (package:integration_test/integration_test.dart:244) flutter: flutter: #14 IntegrationTestWidgetsFlutterBinding.traceTimeline (package:integration_test/integration_test.dart:269) flutter: flutter: #15 IntegrationTestWidgetsFlutterBinding._runAndGetGCInfo (package:integration_test/integration_test.dart:321) flutter: flutter: #16 IntegrationTestWidgetsFlutterBinding.watchPerformance (package:integration_test/integration_test.dart:376) flutter: flutter: #17 main.. (file:///Users/nexus/projects/nevercode/testing_app/integration_test/app_test.dart:22) flutter: flutter: #18 testWidgets.. (package:flutter_test/src/widget_tester.dart:176) flutter: flutter: #19 TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:817) flutter: flutter: 00:02 +0: Testing App Performance Tests Scrolling test [E] flutter: Test failed. See exception logs above. The test description was: Scrolling test flutter: flutter: Consider enabling the flag chain-stack-traces to receive more detailed exceptions. For example, 'dart test --chain-stack-traces'. flutter: 00:02 +0 -1: Testing App Performance Tests Favorites operations test flutter: 00:10 +1 -1: Testing App Performance Tests (tearDownAll) flutter: Warning: integration_test plugin was not detected. If you're running the tests with `flutter drive`, please make sure your tests are in the `integration_test/` directory of your package and use `flutter test $path_to_test` to run it instead. If you're running the tests with Android instrumentation or XCTest, this means that you are not capturing test results properly! See the following link for how to set up the integration_test plugin: https://flutter.dev/docs/testing/integration-tests#testing-on-firebase-test-lab flutter: 00:10 +2 -1: Some tests failed. Failure Details: Failure in method: Scrolling test end of failure 1 ```
abhaysood commented 2 years ago

It fails on Android emulator too - Pixel 3 API 28

logs ``` VMServiceFlutterDriver: Connecting to Flutter application at http://127.0.0.1:56869/pB8P3j2LbFA=/ VMServiceFlutterDriver: Isolate found with number: 651184184411879 VMServiceFlutterDriver: Isolate is paused at start. VMServiceFlutterDriver: Attempting to resume isolate D/eglCodecCommon( 7683): setVertexArrayObject: set vao to 0 (0) 1 0 I/flutter ( 7683): 00:00 +0: Testing App Performance Tests Scrolling test VMServiceFlutterDriver: Connected to Flutter application. I/flutter ( 7683): ══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════ I/flutter ( 7683): The following SocketException was thrown running a test: I/flutter ( 7683): OS Error: Connection refused, errno = 111, address = localhost, port = 60582 I/flutter ( 7683): I/flutter ( 7683): When the exception was thrown, this was the stack: I/flutter ( 7683): #0 _NativeSocket.startConnect (dart:io-patch/socket_patch.dart:677:35) I/flutter ( 7683): #1 _RawSocket.startConnect (dart:io-patch/socket_patch.dart:1680:26) I/flutter ( 7683): #2 RawSocket.startConnect (dart:io-patch/socket_patch.dart:27:23) I/flutter ( 7683): #3 Socket._startConnect (dart:io-patch/socket_patch.dart:1891:22) I/flutter ( 7683): #4 Socket.startConnect (dart:io/socket.dart:738:21) I/flutter ( 7683): #5 _ConnectionTarget.connect (dart:_http/http_impl.dart:2439:18) I/flutter ( 7683): #6 _HttpClient._getConnection.connect (dart:_http/http_impl.dart:2805:12) I/flutter ( 7683): #7 _HttpClient._getConnection (dart:_http/http_impl.dart:2810:12) I/flutter ( 7683): #8 _HttpClient._openUrl (dart:_http/http_impl.dart:2694:12) I/flutter ( 7683): #9 _HttpClient.openUrl (dart:_http/http_impl.dart:2568:7) I/flutter ( 7683): #10 _WebSocketImpl.connect (dart:_http/websocket_impl.dart:1027:24) I/flutter ( 7683): #11 WebSocket.connect (dart:_http/websocket.dart:379:22) I/flutter ( 7683): #12 vmServiceConnectUri (package:vm_service/vm_service_io.dart:32:44) I/flutter ( 7683): #13 IntegrationTestWidgetsFlutterBinding.enableTimeline (package:integration_test/integration_test.dart:244:26) I/flutter ( 7683): I/flutter ( 7683): #14 IntegrationTestWidgetsFlutterBinding.traceTimeline (package:integration_test/integration_test.dart:269:5) I/flutter ( 7683): I/flutter ( 7683): #15 IntegrationTestWidgetsFlutterBinding._runAndGetGCInfo (package:integration_test/integration_test.dart:321:34) I/flutter ( 7683): I/flutter ( 7683): #16 IntegrationTestWidgetsFlutterBinding.watchPerformance (package:integration_test/integration_test.dart:376:43) I/flutter ( 7683): I/flutter ( 7683): #17 main.. (file:///Users/abhaysood/experiments/untitled/integration_test/app_test.dart:18:7) I/flutter ( 7683): I/flutter ( 7683): #18 testWidgets.. (package:flutter_test/src/widget_tester.dart:176:15) I/flutter ( 7683): I/flutter ( 7683): #19 TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:817:5) I/flutter ( 7683): I/flutter ( 7683): I/flutter ( 7683): The test description was: I/flutter ( 7683): Scrolling test I/flutter ( 7683): ════════════════════════════════════════════════════════════════════════════════════════════════════ I/flutter ( 7683): 00:05 +0: Testing App Performance Tests Scrolling test [E] I/flutter ( 7683): Test failed. See exception logs above. I/flutter ( 7683): The test description was: Scrolling test I/flutter ( 7683): I/flutter ( 7683): Consider enabling the flag chain-stack-traces to receive more detailed exceptions. I/flutter ( 7683): For example, 'dart test --chain-stack-traces'. I/flutter ( 7683): 00:05 +0 -1: Testing App Performance Tests Favorites operations test I/flutter ( 7683): 00:14 +1 -1: Testing App Performance Tests (tearDownAll) Failure Details: Failure in method: Scrolling test ══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞═════════════════ The following SocketException was thrown running a test: OS Error: Connection refused, errno = 111, address = localhost, port = 60582 When the exception was thrown, this was the stack: #0 _NativeSocket.startConnect (dart:io-patch/socket_patch.dart:677:35) #1 _RawSocket.startConnect (dart:io-patch/socket_patch.dart:1680:26) #2 RawSocket.startConnect (dart:io-patch/socket_patch.dart:27:23) #3 Socket._startConnect (dart:io-patch/socket_patch.dart:1891:22) #4 Socket.startConnect (dart:io/socket.dart:738:21) #5 _ConnectionTarget.connect (dart:_http/http_impl.dart:2439:18) #6 _HttpClient._getConnection.connect (dart:_http/http_impl.dart:2805:12) #7 _HttpClient._getConnection (dart:_http/http_impl.dart:2810:12) #8 _HttpClient._openUrl (dart:_http/http_impl.dart:2694:12) #9 _HttpClient.openUrl (dart:_http/http_impl.dart:2568:7) #10 _WebSocketImpl.connect (dart:_http/websocket_impl.dart:1027:24) #11 WebSocket.connect (dart:_http/websocket.dart:379:22) #12 vmServiceConnectUri (package:vm_service/vm_service_io.dart:32:44) #13 IntegrationTestWidgetsFlutterBinding.enableTimeline (package:integration_test/integration_test.dart:244:26) #14 IntegrationTestWidgetsFlutterBinding.traceTimeline (package:integration_test/integration_test.dart:269:5) #15 IntegrationTestWidgetsFlutterBinding._runAndGetGCInfo (package:integration_test/integration_test.dart:321:34) #16 IntegrationTestWidgetsFlutterBinding.watchPerformance (package:integration_test/integration_test.dart:376:43) #17 main.. (file:///Users/abhaysood/experiments/untitled/integration_test/app_test.dart:18:7) #18 testWidgets.. (package:flutter_test/src/widget_tester.dart:176:15) #19 TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:817:5) The test description was: Scrolling test ═════════════════════════════════════════════════════════════════ end of failure 1 I/flutter ( 7683): 00:14 +2 -1: Some tests failed. ```
domesticmouse commented 2 years ago

/cc @miquelbeltran

miquelbeltran commented 2 years ago

@domesticmouse I can confirm this issue when running on mobile devices, including emulators, but not as desktop app.

The thing I noticed is that IntegrationTestWidgetsFlutterBinding.watchPerformance is trying to connect to a localhost service, which does not exist on the device but on the local desktop.

In https://github.com/flutter/flutter/issues/90410 the same issue is discussed, and the solution is to add --no-dds to the command, so to run:

flutter drive --driver integration_test/driver.dart --target integration_test/app_test.dart --no-dds
abhaysood commented 2 years ago

I can confirm the codelab works fine when running with --no-dds flag. Thanks @miquelbeltran.

chonghorizons commented 2 years ago

I think this can be closed as a duplicate, then of flutter/flutter#90410. Thanks @miquelbeltran .