Entering text in webview doesn't work

[jeremiahlukus]

Describe the bug

Using https://joyful-noise-staging.joyful-noise.link/users/sign_in as a form I have tried the way mentioned in the docs

      //await $.native.enterTextByIndex('test@hey.com', index: 0);
      await $.native.enterText(
        Selector(text: 'user_email'), // "you@example.com", "Email", etc..
        text: 'bartek@awesome.com',
      );

Nothing shows in form. To Reproduce

Steps to reproduce the behavior:

1.Use https://joyful-noise-staging.joyful-noise.link/users/sign_in as base url

2. try to enter text in form and submit

3. App hangs with no error

Expected behavior

A clear and concise description of what you expected to happen.

Device information

• Device: iPhone14

• Does it also both on physical and virtual devices? [didn't check]

Additional information

flutter --version Flutter 3.7.3 • channel stable • https://github.com/flutter/flutter.git Framework • revision 9944297138 (5 days ago) • 2023-02-08 15:46:04 -0800 Engine • revision 248290d6d5 Tools • Dart 2.19.2 • DevTools 2.20.1 patrol: ^0.10.12 patrol_cli v0.9.4

Additional context

[bartekpacia]

Hi 👋

Did you specify the bundle ID of your app in pubspec.yaml? (see docs for it)

[jeremiahlukus]

Yes i did in the pubspec.yml

patrol:
  app_name: Joyful Noise
  android:
    package_name: com.jparrack.joyful_noise
  ios:
    bundle_id: com.jparrack.joyful-noise

[jeremiahlukus]

Ah i see you are corrrect. it was building com.jparrack.joyful-noise.dev so changing to that worked.

[jeremiahlukus]

@bartekpacia Sorry to keep bugging you it seems it only entered the text that one time. I am unable to enter the text again i have no idea why.

        t =     1.89s         Wait for com.jparrack.joyful-noise.dev to idle
    2023-02-13 18:43:28.105243-0500 RunnerUITests-Runner[61203:11632578] PatrolServer: INFO: submitted 0 dart test results
    2023-02-13 18:43:28.105414-0500 RunnerUITests-Runner[61203:11632578] PatrolServer: INFO: Got 0 dart test results
        t =     3.12s Added attachment named 'Launch Screen'
        t =     3.12s Tear Down
    Test Case '-[RunnerUITestsLaunchTests testLaunch]' passed (3.324 seconds).
    Test Case '-[RunnerUITestsLaunchTests testLaunch]' started.
        t =     0.00s Setting device orientation to Landscape Right
        t =     0.03s     Wait for com.jparrack.joyful-noise.dev to idle
        t =     0.05s Setting appearance mode to Light
    2023-02-13 18:43:29.238648-0500 RunnerUITests-Runner[61203:11632552] PatrolServer: INFO: entering text "wferem1@gmail.com" by index 0...
        t =     0.51s     Get all elements bound by index for: Elements matching predicate 'elementType == 49 OR elementType == 50'
        t =     2.12s     Get number of matches for: Elements matching predicate 'elementType == 49 OR elementType == 50'
        t =     2.12s     Get number of matches for: <XCUIElementQuery: 0x60000058d950> (retry 1)
        t =     2.12s     Get number of matches for: <XCUIElementQuery: 0x60000058d950> (retry 2)
    2023-02-13 18:43:30.854358-0500 RunnerUITests-Runner[61203:11632552] PatrolServer: INFO: found 0 text fields

when i do flutter logs i see

┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄<…>
flutter: \^[[38;5;196m│ ⛔ -------------------------------<…>
flutter: \^[[38;5;196m└───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────<…>
flutter: Patrol (native): enterTextByIndex() started
ocsp responder: (null) did not include status of requested cert
com.apple.WebKit[33418]/1#3 LF=0 trust_evaluate Error Domain=NSOSStatusErrorDomain Code=-34018 "trust_evaluate: com.apple.WebKit[33418]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate" UserInfo={numberOfErrorsDeep=0, NSDescription=trust_evaluate: com.apple.WebKit[33418]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate}
com.apple.WebKit[33418]/1#3 LF=0 trust_evaluate Error Domain=NSOSStatusErrorDomain Code=-34018 "trust_evaluate: com.apple.WebKit[33418]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate" UserInfo={numberOfErrorsDeep=0, NSDescription=trust_evaluate: com.apple.WebKit[33418]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate}
ocsp responder: (null) did not include status of requested cert
com.apple.WebKit[33418]/1#3 LF=0 trust_evaluate Error Domain=NSOSStatusErrorDomain Code=-34018 "trust_evaluate: com.apple.WebKit[33418]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate" UserInfo={numberOfErrorsDeep=0, NSDescription=trust_evaluate: com.apple.WebKit[33418]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate}

(null) did not include status of requested cert. sounds like my error but why is my cert null?

My code is

void main() {
  patrolTest(
    'counter state is the same after going to home and switching apps',
    ($) async {
      main_dart.main();

      await $.pumpAndSettle();
      await $(#signInButtonKey).tap();
      await $.pumpAndSettle();

      logger.e("-------------------------------"); // gets to this line
      await $.native.enterTextByIndex('test@gmail.com', index: 0); //fails here
      logger.e('++++++++++++++++++++++++++++++++++++++++++++');
      await $.native.enterTextByIndex('NyanCar', index: 1);
      await $.pump(Duration(seconds: 2));
      await $.native.enterTextByIndex('heyheyhey', index: 1);
      logger.e('@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@');

    },
    nativeAutomation: true,
  );
}

Ive tried adding

await $.native.enterText(
  Selector(text: 'Email'), 
  text: 'bartek@awesome.com',
);

But it doesnt find the field and errors there.

The form is https://joyful-noise-staging.joyful-noise.link/users/sign_in again.

I feel like this is something to do with using flavors.

Once i get past the webview I can send you a tip for the help if you send me a link.

[bartekpacia]

Thanks for all the details. I'll try to reproduce and fix the issue tomorrow.

I can send you a tip for the help if you send me a link.

I work full-time at @leancodepl on this library, and I'm compensated well :) Thank you, though - this is very kind of you.

[jeremiahlukus]

Great let me know if you need more info.

That is good to hear always good to get paid for working ha

[jeremiahlukus]

Ok i have the bundle identifier RunnerUITests

I created a new debug config so i dont need to pass in the flavor

I saw the target runner is pointing to the correct bundle identifier

in my pubspec

  ios:
    bundle_id: com.jparrack.joyful-noise.RunnerUITests

in my test code

 logger.e("-------------------------------");
 await $.native.enterTextByIndex('test@gmail.com', index: 0, appId: 'com.jparrack.joyful-noise.RunnerUITests');
 logger.e('++++++++++++++++++++++++++++++++++++++++++++');

In flutter logs

flutter: \^[[38;5;196m├┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄<…>
flutter: \^[[38;5;196m│ ⛔ -------------------------------<…>
flutter: \^[[38;5;196m└───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────<…>
flutter: Patrol (native): enterTextByIndex() started
com.apple.WebKit[44776]/1#3 LF=0 trust_evaluate Error Domain=NSOSStatusErrorDomain Code=-34018 "trust_evaluate: com.apple.WebKit[44776]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate" UserInfo={numberOfErrorsDeep=0, NSDescription=trust_evaluate: com.apple.WebKit[44776]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate}
ocsp responder: (null) did not include status of requested cert
com.apple.WebKit[44776]/1#3 LF=0 trust_evaluate Error Domain=NSOSStatusErrorDomain Code=-34018 "trust_evaluate: com.apple.WebKit[44776]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate" UserInfo={numberOfErrorsDeep=0, NSDescription=trust_evaluate: com.apple.WebKit[44776]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate}
com.apple.WebKit[44776]/1#3 LF=0 trust_evaluate Error Domain=NSOSStatusErrorDomain Code=-34018 "trust_evaluate: com.apple.WebKit[44776]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate" UserInfo={numberOfErrorsDeep=0, NSDescription=trust_evaluate: com.apple.WebKit[44776]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate}
ocsp responder: (null) did not include status of requested cert
com.apple.WebKit[44776]/1#3 LF=0 trust_evaluate Error Domain=NSOSStatusErrorDomain Code=-34018 "trust_evaluate: com.apple.WebKit[44776]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate" UserInfo={numberOfErrorsDeep=0, NSDescription=trust_evaluate: com.apple.WebKit[44776]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate}
ocsp responder: (null) did not include status of requested cert
com.apple.WebKit[44776]/1#3 LF=0 trust_evaluate Error Domain=NSOSStatusErrorDomain Code=-34018 "trust_evaluate: com.apple.WebKit[44776]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate" UserInfo={numberOfErrorsDeep=0, NSDescription=trust_evaluate: com.apple.WebKit[44776]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate}

In patrol logs

    2023-02-14 09:26:16.620323-0500 RunnerUITests-Runner[44748:12712462] PatrolServer: INFO: entering text "test@gmail.com" by index 0...
        t =     0.49s     Get all elements bound by index for: Elements matching predicate 'elementType == 49 OR elementType == 50'
        t =     2.40s     Get number of matches for: Elements matching predicate 'elementType == 49 OR elementType == 50'
        t =     2.41s     Get number of matches for: <XCUIElementQuery: 0x60000074f070> (retry 1)
        t =     2.41s     Get number of matches for: <XCUIElementQuery: 0x60000074f070> (retry 2)
    2023-02-14 09:26:18.541099-0500 RunnerUITests-Runner[44748:12712462] PatrolServer: INFO: found 0 text fields
        t =     2.41s     Wait for com.jparrack.joyful-noise.RunnerUITests to idle

The app makes it to the webview and the text field is shown in the screen.

[jeremiahlukus]

after changing my entitlements to

<dict>
    <key>aps-environment</key>
    <string>development</string>
    <key>keychain-access-groups</key>
    <array>
        <string>$(AppIdentifierPrefix)com.jparrack.joyful-noise.RunnerUITests</string>
    </array>
</dict>

flutter: Patrol (native): enterTextByIndex() started
@ClxSimulated, Fix, 1, ll, 37.7858340, -122.4064170, acc, 5.00, course, -1.0, time, 698079021.1, deltaDistance, 0.0, deltaDistanceAccuracy, 1.9, odometer, 0.0, timestampGps, 698079021.1
ocsp responder: (null) did not include status of requested cert
com.apple.WebKit[65079]/1#3 LF=0 trust_evaluate Error Domain=NSOSStatusErrorDomain Code=-34018 "trust_evaluate: com.apple.WebKit[65079]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate" UserInfo={numberOfErrorsDeep=0, NSDescription=trust_evaluate: com.apple.WebKit[65079]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate}

Now it looks like it's at least trying to do it.

[jeremiahlukus]

more logs

flutter: PatrolBinding: Extension stream has no listeners, so host features won't work
flutter: PatrolBinding: registered service extension ext.flutter.patrol
flutter: Patrol (native): Android app name: Joyful Noise
flutter: Patrol (native): iOS app name: Joyful Noise
flutter: Patrol (native): Android package name: com.jparrack.joyful_noise.RunnerUITests
flutter: Patrol (native): iOS bundle identifier: com.jparrack.joyful-noise.RunnerUITests
flutter: 00:00 +0: counter state is the same after going to home and switching apps
@ClxSimulated, Fix, 1, ll, 37.7858340, -122.4064170, acc, 5.00, course, -1.0, time, 698079922.7, deltaDistance, 0.0, deltaDistanceAccuracy, 1.9, odometer, 0.0, timestampGps, 698079922.7
flutter: Patrol (native): configure() started
flutter: Patrol (native): configure() succeeded
Handling kAXUserTestingNotification from AX element pid: 53935, elementOrHash.elementID: 0.1 (0x6000012734d0)
Got user testing notification with payload {
    controllerClass = SBIconController;
    event = ViewDidDisappear;
}
Handling kAXUserTestingNotification from AX element pid: 53935, elementOrHash.elementID: 0.1 (0x6000012734d0)
Calling handler for AX notification kAXUserTestingNotification from AX element pid: 53935, elementOrHash.elementID: 0.1 with payload {
    controllerClass = SBIconController;
    event = ViewDidDisappear;
}
Got user testing notification with payload {
    controllerClass = SBRootFolderController;
    event = ViewDidDisappear;
}
Calling handler for AX notification kAXUserTestingNotification from AX element pid: 53935, elementOrHash.elementID: 0.1 with payload {
    controllerClass = SBRootFolderController;
    event = ViewDidDisappear;
}
Leaf has invalid basic constraints
Leaf has invalid basic constraints
flutter: 00:00 +0: counter state is the same after going to home and switching apps [E]
flutter:   'package:flutter_test/src/binding.dart': Failed assertion: line 903 pos 14: '_pendingExceptionDetails != null': A test overrode FlutterError.onError but either failed to return it to its original state, or had unexpected additional errors that it could not handle. Typically, this is caused by using expect() before restoring FlutterError.onError.
flutter:   dart:core-patch/errors_patch.dart 51:61       _AssertionError._doThrowNew
  dart:core-patch/errors_patch.dart 40:5        _AssertionError._throwNew
  package:flutter_test/src/binding.dart 903:14  TestWidgetsFlutterBinding._runTest.handleUncaughtError
  package:flutter_test/src/binding.dart 908:9   TestWidgetsFlutterBinding._runTest.<fn>
  dart:async/zone.dart 1080:14                  _Zone._processUncaughtError
  dart:async/zone.dart 1284:5                   _CustomZone.handleUncaughtError
  dart:async/future_impl.dart 681:16            Future._propagateToListeners
  dart:async/future_impl.dart 575:5             Future._completeError
  dart:async/zone.dart 1422:47                  _rootRunBinary
  dart:async/zone.dart 1314:19                  _CustomZone.runBinary
  dart:async/future_impl.dart 162:22            _FutureListener.handleError
  dart:async/future_impl.dart 779:47            Future._propagateToListeners.handleError
  dart:async/future_impl.dart 800:13            Future._pr<…>
flutter:   'package:flutter_test/src/binding.dart': Failed assertion: line 1914 pos 12: '_pendingFrame == null': is not true.
flutter:   dart:core-patch/errors_patch.dart 51:61           _AssertionError._doThrowNew
  dart:core-patch/errors_patch.dart 40:5            _AssertionError._throwNew
  package:flutter_test/src/binding.dart 1914:12     LiveTestWidgetsFlutterBinding.postTest
  dart:async/future.dart 302:31                     new Future.sync
  package:test_api/src/backend/invoker.dart 236:18  Invoker.runTearDowns.<fn>.<fn>
  package:test_api/src/backend/invoker.dart 257:15  Invoker._waitForOutstandingCallbacks.<fn>
  dart:async/zone.dart 1398:13                      _rootRun
  dart:async/zone.dart 1300:19                      _CustomZone.run
  dart:async/zone.dart 1803:10                      _runZoned
  dart:async/zone.dart 1746:10                      runZoned
  package:test_api/src/backend/invoker.dart 254:5   Invoker._waitForOutstandingCallbacks
  package:test_api/src/backend/invoker.dart 235:9   Invoker.runTearDowns.<fn>
  dart:async/zone.dart 1398:13                      _rootRun
  dart:async/zone.dart 130<…>
flutter: 00:00 +0 -1: (tearDownAll)
flutter: 00:00 +0 -1: counter state is the same after going to home and switching apps [E]
flutter:   'package:flutter_test/src/binding.dart': Failed assertion: line 902 pos 14: '_parentZone != null': is not true.
flutter:   dart:core-patch/errors_patch.dart 51:61       _AssertionError._doThrowNew
  dart:core-patch/errors_patch.dart 40:5        _AssertionError._throwNew
  package:flutter_test/src/binding.dart 902:14  TestWidgetsFlutterBinding._runTest.handleUncaughtError
  package:flutter_test/src/binding.dart 908:9   TestWidgetsFlutterBinding._runTest.<fn>
  dart:async/zone.dart 1080:14                  _Zone._processUncaughtError
  dart:async/zone.dart 1284:5                   _CustomZone.handleUncaughtError
  dart:async/future_impl.dart 681:16            Future._propagateToListeners
  dart:async/future_impl.dart 575:5             Future._completeError
  dart:async/zone.dart 1422:47                  _rootRunBinary
  dart:async/zone.dart 1314:19                  _CustomZone.runBinary
  dart:async/future_impl.dart 162:22            _FutureListener.handleError
  dart:async/future_impl.dart 779:47            Future._propagateToListeners.handleError
  dart:async/future_impl.dart 800:13            Future._pr<…>
flutter: 00:00 +0 -1: (tearDownAll)
flutter: Sending Dart test results to the native side
flutter: Warning: integration_test plugin was not detected.
flutter:
flutter: If you're running the tests with `flutter drive`, please make sure your tests
flutter: are in the `integration_test/` directory of your package and use
flutter: `flutter test $path_to_test` to run it instead.
flutter:
flutter: If you're running the tests with Android instrumentation or XCTest, this means
flutter: that you are not capturing test results properly! See the following link for
flutter: how to set up the integration_test plugin:
flutter:
flutter: https://flutter.dev/docs/testing/integration-tests#testing-on-firebase-test-lab
flutter:
flutter: 00:00 +1 -1: Some tests failed.
flutter: 00:00 +1 -1: counter state is the same after going to home and switching apps [E]
flutter:   'package:flutter_test/src/binding.dart': Failed assertion: line 902 pos 14: '_parentZone != null': is not true.
flutter:   dart:core-patch/errors_patch.dart 51:61       _AssertionError._doThrowNew
  dart:core-patch/errors_patch.dart 40:5        _AssertionError._throwNew
  package:flutter_test/src/binding.dart 902:14  TestWidgetsFlutterBinding._runTest.handleUncaughtError
  package:flutter_test/src/binding.dart 908:9   TestWidgetsFlutterBinding._runTest.<fn>
  dart:async/zone.dart 1080:14                  _Zone._processUncaughtError
  dart:async/zone.dart 1284:5                   _CustomZone.handleUncaughtError
  dart:async/zone.dart 1210:7                   _CustomZone.runGuarded
  dart:async/zone.dart 1248:23                  _CustomZone.bindCallbackGuarded.<fn>
  dart:async/zone.dart 1398:13                  _rootRun
  dart:async/zone.dart 1300:19                  _CustomZone.run
  dart:async/zone.dart 1208:7                   _CustomZone.runGuarded
  dart:async/zone.dart 1248:23                  _CustomZone.bindCallbackGuarded.<fn>
  dart:async/schedule_microtask.dart 40:21      _microtaskLoop

[bartekpacia]

I haven't yet found time to investigate it further (sorry), but...

It should look like this in the docs:

I doubt it's the cause of the problem, though.

[jeremiahlukus]

Nope i played with that as well both are now Debug and still same issue.

Ill update if i have anything else, i havent made any progress yet.

[jeremiahlukus]

Ok @bartekpacia there was an issue with flutter onError not being restored

Future<void> restoreFlutterError(Future<void> Function() call) async {
  final originalOnError = FlutterError.onError!;
  await call();
  final overriddenOnError = FlutterError.onError!;

  // restore FlutterError.onError
  FlutterError.onError = (FlutterErrorDetails details) {
    if (overriddenOnError != originalOnError) overriddenOnError(details);
    originalOnError(details);
  };
}

void main() {
..
      await restoreFlutterError(() async {
        main_dart.main();
        await $.pumpAndSettle();
      });
      ..
}

This outputs a nice patrol error now

flutter: ══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
flutter: The following assertion was thrown running a test:
flutter: A FocusManager was used after being disposed.
Got system UI orientation (<AXUIElementRef 0x600003f67540> {pid=53935} {uid=[ID:1 hash:0x0]}[kAXApplicationOrientationAttribute]) 4
flutter: Once you have called dispose() on a FocusManager, it can no longer be used.
Preparing screenshot for request: ScreenshotRequest(screenID: 1, rect: nil, orientation: .right, encoding: Encoding(uniformTypeIdentifier: public.jpeg, compressionQuality: 0.700000), options: [.showPointer])
Taking screenshot for request: ScreenshotRequest(screenID: 1, rect: nil, orientation: .right, encoding: Encoding(uniformTypeIdentifier: public.jpeg, compressionQuality: 0.700000), options: [.showPointer])
flutter:
flutter: When the exception was thrown, this was the stack:
flutter: #0      ChangeNotifier.debugAssertNotDisposed.<anonymous closure> (package:flutter/src/foundation/change_notifier.dart:157:9)
flutter: #1      ChangeNotifier.debugAssertNotDisposed (package:flutter/src/foundation/change_notifier.dart:164:6)
flutter: #2      ChangeNotifier.notifyListeners (package:flutter/src/foundation/change_notifier.dart:360:27)
flutter: #3      FocusManager._applyFocusChange (package:flutter/src/widgets/focus_manager.dart:1808:7)
flutter: (elided 10 frames from dart:async)
flutter:
flutter: The test description was:
flutter:   counter state is the same after going to home and switching apps
flutter: ════════════════════════════════════════════════════════════════════════════════════════════════════
flutter: ══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
flutter: The following PatrolActionException was thrown running a test:
flutter: Patrol action failed: GrpcError: enterTextByIndex() failed with code NOT_FOUND (text field at index
flutter: 0 doesn't exist)
flutter:
flutter: When the exception was thrown, this was the stack:
flutter: #0      NativeAutomator._wrapRequest (package:patrol/src/native/native_automator.dart:200:7)
flutter: <asynchronous suspension>
flutter: #1      NativeAutomator.enterTextByIndex (package:patrol/src/native/native_automator.dart:529:5)
flutter: <asynchronous suspension>
flutter: #2      main.<anonymous closure> (file:///Users/jeremiah.parrack/freelance/guitar_tabs/guitar_tabs/integration_test/app_test.dart:37:7)
flutter: <asynchronous suspension>
flutter: #3      patrolTest.<anonymous closure> (package:patrol/src/common.dart:82:7)
flutter: <asynchronous suspension>
flutter: #4      testWidgets.<anonymous closure>.<anonymous closure> (package:flutter_test/src/widget_tester.dart:171:15)
flutter: <asynchronous suspension>
flutter: #5      TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:935:5)
flutter: <asynchronous suspension>
flutter:
flutter: The test description was:
Converting image for request: ScreenshotRequest(screenID: 1, rect: nil, orientation: .right, encoding: Encoding(uniformTypeIdentifier: public.jpeg, compressionQuality: 0.700000), options: [.showPointer])
flutter:   counter state is the same after going to home and switching apps
flutter: ════════════════════════════════════════════════════════════════════════════════════════════════════
flutter:   'package:flutter_test/src/binding.dart': Failed assertion: line 902 pos 14: '_parentZone != null': is not true.
flutter:   dart:core-patch/errors_patch.dart 51:61       _AssertionError._doThrowNew
  dart:core-patch/errors_patch.dart 40:5        _AssertionError._throwNew
  package:flutter_test/src/binding.dart 902:14  TestWidgetsFlutterBinding._runTest.handleUncaughtError
  package:flutter_test/src/binding.dart 908:9   TestWidgetsFlutterBinding._runTest.<fn>
  dart:async/zone.dart 1080:14                  _Zone._processUncaughtError
  dart:async/zone.dart 1284:5                   _CustomZone.handleUncaughtError
  dart:async/future_impl.dart 717:16            Future._propagateToListeners
  dart:async/future_impl.dart 575:5             Future._completeError
  dart:async/zone.dart 1422:47                  _rootRunBinary
  dart:async/zone.dart 1314:19                  _CustomZone.runBinary
  dart:async/future_impl.dart 162:22            _FutureListener.handleError
  dart:async/future_impl.dart 779:47            Future._propagateToListeners.handleError

In a later test you see the same output as before

┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄<…>
flutter: \^[[38;5;196m│ ⛔ -------------------------------<…>
flutter: \^[[38;5;196m└───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────<…>
flutter: Patrol (native): enterTextByIndex() started
ocsp responder: (null) did not include status of requested cert
com.apple.WebKit[42840]/1#3 LF=0 trust_evaluate Error Domain=NSOSStatusErrorDomain Code=-34018 "trust_evaluate: com.apple.WebKit[42840]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate" UserInfo={numberOfErrorsDeep=0, NSDescription=trust_evaluate: com.apple.WebKit[42840]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate}
com.apple.WebKit[42840]/1#3 LF=0 trust_evaluate Error Domain=NSOSStatusErrorDomain Code=-34018 "trust_evaluate: com.apple.WebKit[42840]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate" UserInfo={numberOfErrorsDeep=0, NSDescription=trust_evaluate: com.apple.WebKit[42840]/1#3 LF=0 lacks entitlement com.apple.private.network.socket-delegate}
ocsp responder: (null) did not include status of requested cert

So for some reason this error only triggers on the first test, after that it bypasses this error and gives a less readable error

[jeremiahlukus]

The webview widget is

import 'package:webview_flutter/webview_flutter.dart';

  Widget build(BuildContext context) {
    return Scaffold(
      body: SafeArea(
        child: WebView(
          javascriptMode: JavascriptMode.unrestricted,
          initialUrl: widget.authorizationUrl.toString(),
          onWebViewCreated: (controller) {
            controller.clearCache();
            CookieManager().clearCookies();
          },
          navigationDelegate: (navReq) async {
            if (navReq.url.startsWith(WebAppAuthenticator.redirectUrl().toString())) {
              widget.onAuthorizationCodeRedirectAttempt(
                Uri.parse(navReq.url),
              );
              return NavigationDecision.prevent;
            }
            return NavigationDecision.navigate;
          },
        ),
      ),
    );
  }

So it looks like it's just erroring on the input when the keyboard show up?

[bartekpacia]

I investigated this issue a bit and I'm pretty sure that it comes down to the website being Not Accessibility Friendly.

After going to the webview and taking a native view hierarchy dump, I saw:

<node NAF="true" index="1" text="" resource-id="user_password" class="android.widget.EditText" package="pl.leancode.patrol.example" content-desc="" checkable="false" checked="false" clickable="true" enabled="true" focusable="true" focused="false" scrollable="false" long-clickable="false" password="true" selected="false" bounds="[115,1228][966,1340]" />

and

<node NAF="true" index="1" text="" resource-id="user_email" class="android.widget.EditText" package="pl.leancode.patrol.example" content-desc="" checkable="false" checked="false" clickable="true" enabled="true" focusable="true" focused="true" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[115,989][966,1102]" />

What's important is NAF="true" – this means Not Accessibility Friendly. This is Android-only, but I think it's possible iOS has the same restriction.

Apparently, UIAutomator cannot perform actions on NAF views, even when I'd expect that it should be able to perform them. For example I tried:

await $.native.enterText(
  Selector(resourceId: 'user_email'),
  text: 'charlie@root.me',
);
await $.native.enterText(
  Selector(resourceId: 'user_password'),
  text: 'ny4ncat',
);
await $.native.tap(Selector(text: 'Log in'));

and it doesn't work (output from flutter logs):

I/flutter (28599): ══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
I/flutter (28599): The following PatrolActionException was thrown running a test:
I/flutter (28599): Patrol action failed: GrpcError: enterText() failed with code NOT_FOUND (selector
I/flutter (28599): UiSelector[RESOURCE_ID=user_email] found nothing)

In an act of despair, I also tried listing:

final views = await $.native.getNativeViews(Selector(className: 'android.widget.EditText'));
print("${views.length} views found")
for (final view in views) {
  print('view: $view');
}

and it finds 0 views, while I'd expect 2 to show up (login and password).

So the solution I'd suggest trying out is to make changes to the sign-in website to add "content-description" for accessibility tools so that it won't appear as Not Accessibility Friendly.

See also:

• uiautomatorviewer - What does NAF stand for?
• What is NAF?

Here's the test code and webview screen code that I tried (a slightly modified example app)

[bartekpacia]

I managed to get the test to pass on iOS:

https://user-images.githubusercontent.com/40357511/219218237-a53750fd-f892-423c-bc42-fc56a8f57978.mov

Full code is in #938 - let me know if it works if you git checkout it.

Unfortunately, the code that does it is very ugly (because of hardcoded screen refreshing (a.k.a pumping)):

import 'common.dart';

Future<void> main() async {
  testWebViewA();
}

void testWebViewA() {
  patrol('interacts with the LeanCode website in a webview', ($) async {
    await $.pumpWidgetAndSettle(ExampleApp());

    await $('Open webview screen A').scrollTo().tap();

    // this is a very ugly anti-pattern in tests
    for (var i = 0; i < 300; i++) {
      await $.pump();
    }

    await $.native.enterTextByIndex(
      'barpac02@gmail.com',
      index: 0,
    );
    await $.native.enterTextByIndex(
      'ny4ncat',
      index: 1,
    );
    await $.native.tap(Selector(text: 'Log in'));
  });
}

If I remove the pumping for 300 iterations, I'm getting:

flutter: Patrol (native): enterTextByIndex() failed
flutter: ══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
flutter: The following PatrolActionException was thrown running a test:
flutter: Patrol action failed: GrpcError: enterTextByIndex() failed with code NOT_FOUND (text field at index
flutter: 0 in app pl.leancode.patrol.Example doesn't exist)

There are 2 solutions:

• implement waitUntilVisible() for native views (see #499)

• make $.native.enterTextByIndex() wait if no views are found immediately (just like $.native.enterText does currently)

  You cannot use $.native.enterText because, as I said in the post above, there's no way to refer to these text fields other than by their index. They don't have any kind of accessibility information that'd make it possible to locate them.

What do you think?

[jeremiahlukus]

Ok its not entering text after i remove all the pumpAndSettle's

      await restoreFlutterError(() async {
        main_dart.main();
        // await $.pumpAndSettle();
      });
     // await $.pumpAndSettle();
      for (var i = 0; i < 300; i++) {
        await $.pump();
      }
      await $(#signInButtonKey).tap();
      // await $.pumpAndSettle();
      // this is a very ugly anti-pattern in tests
      for (var i = 0; i < 300; i++) {
        await $.pump();
      }
      await $.native.enterTextByIndex('test@gmail.com', index: 0);
      await $.native.enterTextByIndex('NyanCar', index: 1);

Now its not closing the dialog and adding a "q" at the end of the email for some reason. This is copy paste from your branch

         await $.native.enterTextByIndex(
        'barpac02@gmail.com',
        index: 0,
      );
      await $.native.enterTextByIndex(
        'ny4ncat',
        index: 1,
      );
      await $.native.tap(Selector(text: 'Log in'));

just double checked landscape isnt broken in the app ha

[bartekpacia]

Now its not closing the dialog

What dialog?

and adding a "q" at the end of the email for some reason.

Wow, this is sooo weird. Doesn't occur on my machine.

its not entering text after i remove all the pumpAndSettle's

Yeah. I think adding the timeout to enterTextByIndex() is the best way to solve this.

[bartekpacia]

and adding a "q" at the end of the email for some reason.

I think it's because you're in landscape mode.

After Patrol enters the email into the first text field, it taps on the second text field to enter the password. But, the keyboard is obscuring the second text field, so Patrol taps on the keyboard, specifically on the q character :)

That's just my guess.

Try portrait mode and let me know if the issue persists.

[jeremiahlukus]

Is there a way to force run in portrait?
My app supports landscape so removing it from deployment info just to run tests isn't something i can do.

Something like

exec gcloud firebase test ios run \
  --test "build/ios_integ/Build/Products/ios_tests.zip" \
  --device model=iphone8,version=15.7,locale=en_US,orientation=portrait

For the patrol cli?

[jeremiahlukus]

after changing to only portrait it started working.

[bartekpacia]

Portrait/landscape mode is a setting belonging to the iOS Simulator app, you should change it there.

Maybe you have your Simulator set to run by default in landscape mode? I don't know, really.

[jeremiahlukus]

In the file RunnerUILaunchTests it contains

+ (BOOL)runsForEachTargetApplicationUIConfiguration {
    return YES;
}

which looks like its doing it.

Back to the webview now its entering text however the login button isnt being pressed

   nans         Checking existence of `"Log in" Any`
    2023-02-16 08:37:02.148338-0500 RunnerUITests-Runner[37964:15537629] PatrolServer: INFO: found view with text "Log in", will tap on it
        t =      nans Find the "Log in" Any
        t =      nans Tap "Log in" Other[0.00, 0.00]
        t =      nans     Wait for com.jparrack.joyful-noise.RunnerUITests to idle
        t =      nans     Find the "Log in" Other
        t =      nans     Check for interrupting elements affecting "Log in" Other
        t =      nans     Synthesize event
        t =      nans         Find the "Log in" Other
        t =      nans     Wait for com.jparrack.joyful-noise.RunnerUITests to idle
    2023-02-16 08:37:02.815215-0500 RunnerUITests-Runner[37964:15537629] PatrolServer: INFO: done tapping on view with text "Log in"

I think the keyboard is covering up the button is there a way to manually close the keyboard? Even in the video you sent you should have seen an error in the webview after clicking on submit.

[jeremiahlukus]

adding await $.native.tap(Selector(text: 'Done')); did close the keyboard but didnt click on login.

Now the output is

    2023-02-16 08:48:47.115358-0500 RunnerUITests-Runner[42202:15559199] PatrolServer: INFO: waiting for existence of view with text "Log in"
        t =      nans Waiting 10.0s for "Log in" Any to exist
        t =      nans     Checking `Expect predicate `exists == 1` for object "Log in" Any`
        t =      nans         Checking existence of `"Log in" Any`
    2023-02-16 08:48:48.215951-0500 RunnerUITests-Runner[42202:15559199] PatrolServer: INFO: found view with text "Log in", will tap on it
        t =      nans Find the "Log in" Any
        t =      nans Tap "Log in" Other
        t =      nans     Wait for com.jparrack.joyful-noise.RunnerUITests to idle
        t =      nans     Find the "Log in" Other
        t =      nans     Check for interrupting elements affecting "Log in" Other
        t =      nans     Synthesize event
        t =      nans     Wait for com.jparrack.joyful-noise.RunnerUITests to idle
    2023-02-16 08:48:48.671168-0500 RunnerUITests-Runner[42202:15559199] PatrolServer: INFO: done tapping on view with text "Log in"
    2023-02-16 08:48:48.671422-0500 RunnerUITests-Runner[42202:15559199] PatrolServer: INFO: result: ()
    2023-02-16 08:48:48.806536-0500 RunnerUITests-Runner[42202:15560051] PatrolServer: INFO: submitted 0 dart test results
    2023-02-16 08:48:48.806671-0500 RunnerUITests-Runner[42202:15560051] PatrolServer: INFO: Got 0 dart test results
    Test Suite 'All tests' started at 2023-02-16 08:48:49.683

[bartekpacia]

In the file RunnerUILaunchTests it contains

Delete this file – it's not needed.

but didnt click on login.

That's because you've got 2 "Log in" texts - the first one is the heading, and the second one is on the button.

This should be possible to do with:

await $.native.tap(
      Selector(
        text: 'Log in',
        instance: 1,
      ),
    );

Unfortunately it isn't, but I see this is very important. Going to fix it as part of #663.

[jeremiahlukus]

Ok deleting that file was a good idea, now its running the tests once instead of multiple time with different orientations and dark mode/light mode.

Great thanks for working on that already.

[jeremiahlukus]

I changed the text to "Sign in" to be unique however i still have issues clicking on the button

     await $.native.tap(
        Selector(
          text: 'Sign in',
        ),
      );

And still unable to click on it.

[jeremiahlukus]

     // works
      await $.native.enterTextByIndex(
        'test@hey.com',
        index: 0,
      );

         // works
      await $.native.enterTextByIndex(
        'testing',
        index: 1,
      );
      logger.e('---------------------------------');
         // works
      await $.native.tap(Selector(text: 'Done'));
      for (var i = 0; i < 300; i++) {
        await $.pump();
      }
         // broken
      await $.native.tap(Selector(text: 'Sign in'));

      logger.e('@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@');
      for (var i = 0; i < 300; i++) {
        await $.pump();
      }
       // broken
      await $.native.tap(Selector(text: 'Sign in'));
      for (var i = 0; i < 300; i++) {
        await $.pump();
      }

[bartekpacia]

import 'common.dart';

Future<void> main() async {
  testWebViewD();
}

void testWebViewD() {
  patrol('interacts with the login form website in a webview', ($) async {
    await $.pumpWidgetAndSettle(ExampleApp());

    await $('Open webview (login form)').scrollTo().tap();

    await $.native.enterTextByIndex('test@hey.com', index: 0);
    await $.native.enterTextByIndex('some pass', index: 1);
    await $.native.tap(Selector(text: 'Sign in'));
  });
}

This reliably works on my iPhone 14 simulator (I'm using master branch of patrol):

https://user-images.githubusercontent.com/40357511/219620134-e1690b95-2eaa-4992-8265-29bbf90caf32.mov

[jeremiahlukus]

Ok finally got it flutter integration tests really deosnt like it when you overwrite onError.

If you do you need to wrap your app in

Future<void> restoreFlutterError(Future<void> Function() call) async {
  final originalOnError = FlutterError.onError!;
  await call();
  final overriddenOnError = FlutterError.onError!;

  // restore FlutterError.onError
  FlutterError.onError = (FlutterErrorDetails details) {
    if (overriddenOnError != originalOnError) overriddenOnError(details);
    originalOnError(details);
  };
}

void main() {
  patrolTest(
    'counter state is the same after going to home and switching apps',
    ($) async {
      await restoreFlutterError(() async {
        main_dart.main();
      });
      for (var i = 0; i < 300; i++) {
        await $.pump();
      }
      // if you make a call that might trigger an error 
      await restoreFlutterError(() async {
        await $.native.tap(Selector(text: 'Sign in'));
      });

I thought wrapping the app with restoreFlutterError would wrap all errors however anytime there is an error the app will break if you dont wrap that call in restoreFlutterError()

thanks for all your help closing this now.

[bartekpacia]

Thanks a lot, we'll be playing with FlutterError.onError in #951 so that's definitely useful info :) thanks!

[jeremiahlukus]

Doing this is better

https://github.com/flutter/flutter/issues/34499

final FlutterExceptionHandler? originalOnError = FlutterError.onError;
     main_dart.main();
           for (var i = 0; i < 300; i++) {
        await $.pump();
      }
FlutterError.onError = originalOnError;
...
expect()

The other way will load the app but will error out on the expect(). @bartekpacia

[github-actions[bot]]

This issue has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar problem, please file a new issue. Make sure to follow the template and provide all the information necessary to reproduce the issue.