Closed findms closed 1 year ago
bartekpacia
commented
1 year ago I cannot reproduce this issue. No repository and reproduction steps were provided.
What's more, I see a call to IntegrationTestWidgetsFlutterBinding.ensureInitialized() the screenshot. You cannot initialize the bindings yourself - Patrol does this.
findms
commented
1 year ago @bartekpacia This is not related to any code issue but just like a issue with the setup or how patrol need to be configured. So in any repo just make nativeAutomation flag to true and then on debugging this error returns.
Also, Can we go ahead and remove this IntegrationTestWidgetsFlutterBinding.ensureInitialized() in listener.dart which is inside flutter sdk ? Not sure if this is allowed even.
bartekpacia
commented
1 year ago This is not related to any code issue but just like a issue with the setup or how patrol need to be configured. So in any repo just make nativeAutomation flag to true and then on debugging this error returns.
I know many Patrol users, both internal and external, and nobody is reporting this problem. This is something specific to your particular project/machine/setup, that's why I'm asking for full reproduction info. Otherwise I just can't help you.
Also, Can we go ahead and remove this IntegrationTestWidgetsFlutterBinding.ensureInitialized() in listener.dart which is inside flutter sdk
No file named listener.dart exists in the Flutter SDK.
listener.dart is generated automatically when you run flutter test integration_test. This issue might be explain this in more detail.
Patrol CLI doesn't interact with listener.dart in any way. It's very strange that this file gets executed.
findms
commented
1 year ago Thanks @bartekpacia for quick input. Not sure about flutter test but I am using patrol test. Also when I tried removing ensureInitialization still the error is same as this gets overridden.
I will look more into my project/machine/setup if I can rectify this, then i will go ahead to close this issue. Else I will share a sample repo to have a look.
bartekpacia
commented
1 year ago Could you post output of patrol test --verbose?
findms
commented
1 year ago This is getting logged with patrol test --verbose where after performing tap() on webview it throws an exception as pumpAndSettle timed out
Note: I face this issue when I run my tests with patrol test which I have already logged here https://github.com/leancodepl/patrol/issues/1332 but this issue #1380 I get when I debug my code. Apologies I could not share the whole log as it has some sensitive information.
`2023-06-20 16:31:46.973836+0200 RunnerUITests-Runner[83990:3603800] PatrolServer: INFO: submitted 1 dart test results
2023-06-20 16:31:46.974309+0200 RunnerUITests-Runner[83990:3603800] PatrolServer: INFO: Got 1 dart test results
Test Suite 'All tests' started at 2023-06-20 16:31:47.405
Test Suite 'RunnerUITests.xctest' started at 2023-06-20 16:31:47.406
Test Suite 'RunnerUITests' started at 2023-06-20 16:31:47.407
Test Case '-[RunnerUITests testInitialTest]' started.
t = 0.00s Start Test at 2023-06-20 16:31:47.408
t = 0.04s Set Up
/Users/--------/---------/------------------------------/packages/app/ios/RunnerUITests/RunnerUITests.m:4: error: -[RunnerUITests testInitialTest] : ((success) is true) failed - ══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞═════════════════
The following assertion was thrown running a test:
pumpAndSettle timed out
When the exception was thrown, this was the stack:
#0 WidgetTester.pumpAndSettle.<anonymous closure> (package:flutter_test/src/widget_tester.dart:674:11)
<asynchronous suspension>
#1 TestAsyncUtils.guard.<anonymous closure> (package:flutter_test/src/test_async_utils.dart:117:7)
<asynchronous suspension>
The test description was:
initial test
═════════════════════════════════════════════════════════════════
t = 0.08s Tear Down
Test Case '-[RunnerUITests testInitialTest]' failed (0.288 seconds).
Test Suite 'RunnerUITests' failed at 2023-06-20 16:31:47.696.
Executed 1 test, with 1 failure (0 unexpected) in 0.288 (0.289) seconds
Test Suite 'RunnerUITests.xctest' failed at 2023-06-20 16:31:47.697.
Executed 1 test, with 1 failure (0 unexpected) in 0.288 (0.290) seconds
Test Suite 'All tests' failed at 2023-06-20 16:31:47.697.
Executed 1 test, with 1 failure (0 unexpected) in 0.288 (0.292) seconds`
juanleondev
commented
1 year ago I'm getting the same error, maybe is something related to the last update? 🤔
bartekpacia
commented
1 year ago Hi @juanleondev 👋🏻 Thanks for reporting that you also experience this problem.
Can you post full patrol test --verbose logs? That'd help in debugging.
juanleondev
commented
1 year ago Hi @bartekpacia ! Sure, these are the logs:
00:22 +0 -1: loading /Users/jjleoncamilo/Documents/development/somnio/Queerie/queerie-app/integration_test/scenarios/onboarding_scenario_test.dart [E]
Failed to load "/Users/jjleoncamilo/Documents/development/somnio/Queerie/queerie-app/integration_test/scenarios/onboarding_scenario_test.dart": 'package:flutter/src/foundation/binding.dart': Failed assertion: line 150 pos 12: '_debugInitializedType == null': Binding is already initialized to IntegrationTestWidgetsFlutterBinding
dart:core _AssertionError._throwNew
package:flutter/src/foundation/binding.dart 150:12 new BindingBase
package:flutter_test/src/binding.dart new _TestWidgetsFlutterBinding&BindingBase&SchedulerBinding
package:flutter_test/src/binding.dart new _TestWidgetsFlutterBinding&BindingBase&SchedulerBinding&ServicesBinding
package:flutter_test/src/binding.dart new _TestWidgetsFlutterBinding&BindingBase&SchedulerBinding&ServicesBinding&GestureBinding
package:flutter_test/src/binding.dart new _TestWidgetsFlutterBinding&BindingBase&SchedulerBinding&ServicesBinding&GestureBinding&SemanticsBinding
package:flutter_test/src/binding.dart new _TestWidgetsFlutterBinding&BindingBase&SchedulerBinding&ServicesBinding&GestureBinding&SemanticsBinding&RendererBinding
package:flutter_test/src/binding.dart new _TestWidgetsFlutterBinding&BindingBase&SchedulerBinding&ServicesBinding&GestureBinding&SemanticsBinding&RendererBinding&PaintingBinding
package:flutter_test/src/binding.dart new _TestWidgetsFlutterBinding&BindingBase&SchedulerBinding&ServicesBinding&GestureBinding&SemanticsBinding&RendererBinding&PaintingBinding&WidgetsBinding
package:flutter_test/src/binding.dart new _TestWidgetsFlutterBinding&BindingBase&SchedulerBinding&ServicesBinding&GestureBinding&SemanticsBinding&RendererBinding&PaintingBinding&WidgetsBinding&TestDefaultBinaryMessengerBinding
package:flutter_test/src/binding.dart new TestWidgetsFlutterBinding
package:flutter_test/src/binding.dart new LiveTestWidgetsFlutterBinding
package:integration_test/integration_test.dart new IntegrationTestWidgetsFlutterBinding
package:patrol/src/binding.dart new PatrolBinding
package:patrol/src/binding.dart 77:7 new PatrolBinding.ensureInitialized
package:patrol/src/common.dart 54:39 patrolTest
Users/jjleoncamilo/Documents/development/somnio/Queerie/queerie-app/integration_test/scenarios/onboarding_scenario_test.dart 24:3 main
[ +100 ms] test 0: Test harness is no longer needed by test process
[ ] test 0: finished
[ +1 ms] test 0: cleaning up...
[ ] test 0: ensuring test device is terminated.
[ ] executing: /Users/jjleoncamilo/Library/Android/sdk/platform-tools/adb -s emulator-5554 shell am force-stop com.quute.queerie.dev
[ ] test 0: Test process is no longer needed by test harness
To run this test again: /Users/jjleoncamilo/flutter/bin/cache/dart-sdk/bin/dart test /Users/jjleoncamilo/Documents/development/somnio/Queerie/queerie-app/integration_test/scenarios/onboarding_scenario_test.dart -p vm --plain-name 'loading /Users/jjleoncamilo/Documents/development/somnio/Queerie/queerie-app/integration_test/scenarios/onboarding_scenario_test.dart'
00:22 +0 -1: Some tests failed.
[ +3 ms] test package returned with exit code 1
[ ] Runtime for phase TestRunner: Wall-clock: 0:00:22.611295; combined: 0:00:22.611320.
[ ] Runtime for phase Compile: Wall-clock: 0:00:00.000000; combined: 0:00:00.000000.
[ ] Runtime for phase Run: Wall-clock: 0:00:22.518105; combined: 0:00:22.518127.
[ ] Runtime for phase CoverageTotal: Wall-clock: 0:00:00.000000; combined: 0:00:00.000000.
[ ] Runtime for phase CoverageCollect: Wall-clock: 0:00:00.000000; combined: 0:00:00.000000.
[ ] Runtime for phase CoverageParseJson: Wall-clock: 0:00:00.000000; combined: 0:00:00.000000.
[ ] Runtime for phase CoverageAddHitmap: Wall-clock: 0:00:00.000000; combined: 0:00:00.000000.
[ ] Runtime for phase CoverageDataCollect: Wall-clock: 0:00:00.000000; combined: 0:00:00.000000.
[ ] Runtime for phase WatcherFinishedTest: Wall-clock: 0:00:00.000962; combined: 0:00:00.000963.
[ ] "flutter test" took 25,749ms.
[ +1 ms]
#0 throwToolExit (package:flutter_tools/src/base/common.dart:10:3)
#1 TestCommand.runCommand (package:flutter_tools/src/commands/test.dart:504:7)
<asynchronous suspension>
#2 FlutterCommand.run.<anonymous closure> (package:flutter_tools/src/runner/flutter_command.dart:1336:27)
<asynchronous suspension>
#3 AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:150:19)
<asynchronous suspension>
#4 CommandRunner.runCommand (package:args/command_runner.dart:212:13)
<asynchronous suspension>
#5 FlutterCommandRunner.runCommand.<anonymous closure> (package:flutter_tools/src/runner/flutter_command_runner.dart:296:9)
<asynchronous suspension>
#6 AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:150:19)
<asynchronous suspension>
#7 FlutterCommandRunner.runCommand (package:flutter_tools/src/runner/flutter_command_runner.dart:244:5)
<asynchronous suspension>
#8 run.<anonymous closure>.<anonymous closure> (package:flutter_tools/runner.dart:91:9)
<asynchronous suspension>
#9 AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:150:19)
<asynchronous suspension>
#10 main (package:flutter_tools/executable.dart:91:3)
<asynchronous suspension>
[ +110 ms] executing: /Users/jjleoncamilo/Library/Android/sdk/platform-tools/adb -s emulator-5554 uninstall com.quute.queerie.dev
[ +83 ms] ensureAnalyticsSent: 192ms
[ ] Running 1 shutdown hook
[ ] Shutdown hooks complete
[ ] exiting with code 1@bartekpacia Sorry, I was using fluter test, and it was generating the Binding error.
findms
commented
1 year ago @juanleondev I am using patrol test but getting this error while debugging. Not sure how to use patrol test in launch.json while debugging via vscode.
bartekpacia
commented
1 year ago @findmanish Please provide full logs of running patrol test --verbose. What you provided above in this comment is not full logs. I want to see something like this (just an example):
$ patrol test --verbose
You're using Patrol CLI 2.0, which has breaking changes.
Read the migration guide at https://patrol.leancode.co/v2.
Disable this warning by setting the PATROL_MIGRATED environment variable.
Verbose mode enabled. More logs will be printed.
Received 16 test target(s)
Received test target: /Users/bartek/dev/leancode/patrol/packages/patrol/example/integration_test/android_app_test.dart
Received test target: /Users/bartek/dev/leancode/patrol/packages/patrol/example/integration_test/example_test.dart
Received test target: /Users/bartek/dev/leancode/patrol/packages/patrol/example/integration_test/notifications_test.dart
Received test target: /Users/bartek/dev/leancode/patrol/packages/patrol/example/integration_test/open_app_test.dart
Received test target: /Users/bartek/dev/leancode/patrol/packages/patrol/example/integration_test/open_quick_settings_test.dart
Received test target: /Users/bartek/dev/leancode/patrol/packages/patrol/example/integration_test/permissions_location_test.dart
Received test target: /Users/bartek/dev/leancode/patrol/packages/patrol/example/integration_test/permissions_many_test.dart
Received test target: /Users/bartek/dev/leancode/patrol/packages/patrol/example/integration_test/service_airplane_mode_test.dart
Received test target: /Users/bartek/dev/leancode/patrol/packages/patrol/example/integration_test/service_bluetooth_test.dart
Received test target: /Users/bartek/dev/leancode/patrol/packages/patrol/example/integration_test/service_cellular_test.dart
Received test target: /Users/bartek/dev/leancode/patrol/packages/patrol/example/integration_test/service_dark_mode_test.dart
Received test target: /Users/bartek/dev/leancode/patrol/packages/patrol/example/integration_test/service_wifi_test.dart
Received test target: /Users/bartek/dev/leancode/patrol/packages/patrol/example/integration_test/swipe_test.dart
Received test target: /Users/bartek/dev/leancode/patrol/packages/patrol/example/integration_test/webview_hackernews_test.dart
Received test target: /Users/bartek/dev/leancode/patrol/packages/patrol/example/integration_test/webview_leancode_test.dart
Received test target: /Users/bartek/dev/leancode/patrol/packages/patrol/example/integration_test/webview_stackoverflow_test.dart
Bundled 16 test(s) in /Users/bartek/dev/leancode/patrol/packages/patrol/example/integration_test/test_bundle.dart
$ flutter --no-version-check devices --machine
Error: No devices attached
#0 throwToolExit (package:patrol_cli/src/base/exceptions.dart:7:3)
#1 DeviceFinder.findDevicesToUse (package:patrol_cli/src/devices.dart:83:7)
#2 DeviceFinder.find (package:patrol_cli/src/devices.dart:54:12)
<asynchronous suspension>
#3 TestCommand.run (package:patrol_cli/src/commands/test.dart:125:21)
<asynchronous suspension>
#4 CommandRunner.runCommand (package:args/command_runner.dart:212:13)
<asynchronous suspension>
#5 PatrolCommandRunner.runCommand (package:patrol_cli/src/runner/patrol_command_runner.dart:314:18)
<asynchronous suspension>
#6 PatrolCommandRunner.run (package:patrol_cli/src/runner/patrol_command_runner.dart:258:18)
<asynchronous suspension>
#7 patrolCommandRunner (package:patrol_cli/src/runner/patrol_command_runner.dart:73:20)
<asynchronous suspension>
#8 main (file:///Users/bartek/dev/leancode/patrol/packages/patrol_cli/bin/main.dart:6:20)
<asynchronous suspension>@bartekpacia THis is I got in 1 run.Hope this helps. Just to add more. It works fine while running but I am facing this error while debugging via vscode inbuild run and debug. IS there any where to debug my code apart from this vscode where it is therowing this error? ANy debuggin documentation where I can stop my code in case of wrong locator or any change in method IfI have to perform. Currently Everytime I have to run my code if it works fine or not.
bartekpacia
commented
1 year ago It works fine while running but I am facing this error while debugging via vscode inbuild run and debug. IS there any where to debug my code apart from this vscode where it is therowing this error?
Patrol has no integration with "Build & Debug" in VSCode, at least not now. To see logs, you can use e.g. adb logcat. For developing integration tests, using patrol develop might be a good idea.
Closing because this is not a problem with Patrol.
findms
commented
1 year ago @bartekpacia But it works fine for same when I use nativaeAutomation:false and able to debug without any problem..? any possible reason due to nativeAutomation flag?
bartekpacia
commented
1 year ago nativeAutomation: true requires you to use Patrol CLI, because there's a lot more to it than just a simple call to flutter test. Patrol CLI takes care of all the setup.
When nativeAutomation is not passed (so it defaults to false), then your test is a normal widget test, and can be run with flutter test, and can be run from VSCode as well.
I hope I helped you.
I also recommend you read the documentation. If you find something is missing from it, please raise an issue or even better, make a PR and add/improve something :)
findms
commented
1 year ago Thank you @bartekpacia ..My nativeAutomation setup is correct as it is running fine. I am just stuck how to debug it in case if I put a wrong locator which doesn't exist or a function which is not working as expected and have no idea why?..thats the only things I wanted to ask..Thank you for your time. Good Day.
bartekpacia
commented
1 year ago For debugging tests (e.g. when you have a wrong locator), the patrol develop command is nice. I recommend you check it out.
When running with patrol test, you can also open a new terminal tab and run flutter logs - it'll forward logs and you'll be able to see what's happening inside the tests, be able to see print()s, see exceptions that are thrown, etc.
findms
commented
1 year ago @bartekpacia This is still a VALID and annoying issue where anyone using patrol with nativeAutomation:true can not perform debugging.. No matter if it's VSCode or AndroidStudio due to this same error right?
bartekpacia
commented
1 year ago We do not support debugging in Android Studio and in VSCode. Therefore, it's not a bug.
github-actions[bot]
commented
11 months ago This issue has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar problem, please file a new issue. Make sure to follow the template and provide all the information necessary to reproduce the issue.
Steps to reproduce
While debugging the tests keeping nativaAutomation:true, app stuck on spash screen and below eeror is being thrown in lisner.dart which is inside flutter sdk
If I keep nativeAutomation:false, this problem doesn't occur.
Tests are being run as
patrol test --target integration_testA previously closed issue has some details: https://github.com/leancodepl/patrol/issues/854
Actual results
While debugging the tests keeping nativaAutomation:true, app stuck on spash screen and below eeror is being thrown in lisner.dart which is inside flutter sdk
If I keep nativeAutomation:false, this problem doesn't occur.
Logs
Logs
``` ```Patrol version
patrol: 1.1.11
Patrol Doctor output
Patrol Doctor output
``` Verbose mode enabled. More logs will be printed. Patrol CLI version: 1.1.11 Program adb found in /Users/--------/Library/Android/sdk//platform-tools//adb Env var $ANDROID_HOME set to /Users/--------/Library/Android/sdk/ Program xcodebuild found in /usr/bin/xcodebuild Program ideviceinstaller found in /opt/homebrew/bin/ideviceinstaller Program ios-deploy found in /opt/homebrew/bin/ios-deploy ```Flutter Doctor output
Flutter Doctor output
``` Doctor summary (to see all details, run flutter doctor -v): [✓] Flutter (Channel stable, 3.7.7, on macOS 13.4 22F66 darwin-arm64, locale en-DE) [✓] Android toolchain - develop for Android devices (Android SDK version 33.0.2) [✓] Xcode - develop for iOS and macOS (Xcode 14.3) [✓] Chrome - develop for the web [✓] Android Studio (version 2022.2) [✓] VS Code (version 1.78.2) [✓] Connected device (3 available) [✓] HTTP Host Availability ```