[🐛 Bug]: send_keys has no effect in textbox flutter

[jay-athelas]

What happened?

I am trying to automate an e2e flow using appium selenium and XCUITest driver. I am able to click into elements but not writing text into inputs

the test is mean to run in safari browser on iOS device, the flutter element is FormBuilderTextField.

this is the test

// ignore_for_file: use_build_context_synchronously

import 'package:email_validator/email_validator.dart';
import 'package:flutter/foundation.dart';
import 'package:flutter/material.dart';
import 'package:flutter_form_builder/flutter_form_builder.dart';
import 'package:scribe/components/agreements.dart';
import 'package:scribe/components/windowControlsWrapper.dart';
import 'package:scribe/screens/appointmentScreen.dart';
import 'package:scribe/screens/recordingScreen.dart';
import 'package:scribe/services/segment/segment.dart';
import 'package:scribe/utils/constants.dart';
import 'package:scribe/components/popups.dart';
import 'package:scribe/qr_view_example.dart';
import 'package:scribe/services/auth.dart';
import 'package:scribe/services/featureFlag.dart';
import 'package:scribe/services/rateLimiting.dart';
import 'package:universal_platform/universal_platform.dart';

/////////////////////////////////////////////////////////////////////////////////////////
//                                                                                     //
//  DEPRECATION WARNING: This Flutter screen is scheduled for deprecation.             //
//  Please consider using or migrating to a new screen setup as this will be           //
//  removed in future updates.                                                         //
//                                                                                     //
/////////////////////////////////////////////////////////////////////////////////////////

class LoginScreen extends StatefulWidget {
  const LoginScreen({super.key});

  @override
  State<LoginScreen> createState() => _LoginScreenState();
}

class _LoginScreenState extends State<LoginScreen> {
  final _formKey = GlobalKey<FormBuilderState>();

  bool isLogin = false;

  bool showOTP = false;

  TextEditingController otpTextField = TextEditingController();

  @override
  Widget build(BuildContext context) {
    return Scaffold(
        appBar: customAppBar,
        body: Padding(
          padding: const EdgeInsets.fromLTRB(10, 20, 10, 10),
          child: FormBuilder(
            key: _formKey,
            child: Column(
              mainAxisAlignment: MainAxisAlignment.start,
              children: [
                Text(
                  isLogin ? "Login" : "Sign Up",
                  style: const TextStyle(
                      fontSize: 36, fontWeight: FontWeight.bold),
                  textAlign: TextAlign.center,
                ),
                const SizedBox(height: 30),
                Row(
                  children: [
                    Flexible(
                      flex: 2,
                      child: FormBuilderTextField(
                        key: const ValueKey("email_input"),
                        name: "email",
                        decoration: const InputDecoration(
                            label: Text("Email"),
                            contentPadding: EdgeInsets.fromLTRB(15, 15, 15, 15),
                            border: OutlineInputBorder(
                                borderRadius:
                                    BorderRadius.all(Radius.circular(5)))),
                      ),
                    ),
                    Flexible(
                      flex: 1,
                      child: Padding(
                        padding: const EdgeInsets.fromLTRB(10, 0, 5, 0),
                        child: ElevatedButton.icon(
                            key: const ValueKey("send_email_button"),
                            style: ElevatedButton.styleFrom(
                                minimumSize: const Size(double.infinity, 20),
                                foregroundColor: Colors.white,
                                backgroundColor: scribePurple,
                                padding: const EdgeInsets.all(15),
                                shape: const RoundedRectangleBorder(
                                    borderRadius:
                                        BorderRadius.all(Radius.circular(5)))),
                            onPressed: () async {
                              Map<String, dynamic> formValues = _formKey
                                  .currentState!.fields
                                  .map((key, field) =>
                                      MapEntry(key, field.value));

                              String email =
                                  formValues['email'].toString().toUpperCase();

                              if (!EmailValidator.validate(email)) {
                                showSnackbar(
                                    context: context,
                                    message: "Invalid Email format",
                                    type: SnackbarMessageType.BAD);
                                return;
                              }

                              AuthStatus status;

                              if (isLogin) {
                                showFullScreenLoader(context: context);

                                status = await AuthService
                                    .sendEmailForPasswordlessVerification(
                                        email: email,
                                        context: context,
                                        useOTP:
                                            FeatureFlag.full_screen_simple_otp);
                                closeLoader(context: context);
                                if (status == AuthStatus.EXPIRED) {
                                  showRateLimitingPopup(
                                      context: context,
                                      type: Popuptype.ACCOUNT_EXPIRED,
                                      email: email);
                                }
                              } else {
                                print(
                                    "[Signup Mode] Created new scribe account and sent email code");

                                showFullScreenLoader(context: context);

                                status =
                                    await AuthService.createNewScribeAccount(
                                        email: email, context: context);

                                closeLoader(context: context);
                              }
                              if (status == AuthStatus.OK) {
                                Segment.trackSignup(email);
                                setState(() {
                                  showOTP = true;
                                });
                              }
                            },
                            icon: const Icon(Icons.send),
                            label: const Text("Send")),
                      ),
                    )
                  ],
                ),
                const SizedBox(height: 20),
                showOTP
                    ? Row(
                        children: [
                          Flexible(
                            flex: 2,
                            child: FormBuilderTextField(
                              key: const ValueKey("otp_input"),
                              name: "otp",
                              controller: otpTextField,
                              decoration: InputDecoration(
                                  label: const Text("OTP Password"),
                                  contentPadding:
                                      const EdgeInsets.fromLTRB(15, 15, 15, 15),
                                  border: const OutlineInputBorder(
                                      borderRadius:
                                          BorderRadius.all(Radius.circular(5))),
                                  suffixIcon: kIsWeb
                                      ? null
                                      : TextButton.icon(
                                          onPressed: () async {
                                            String qrCode = await showDialog(
                                                barrierDismissible: true,
                                                context: context,
                                                builder:
                                                    (BuildContext context) {
                                                  return const QRScanScreen();
                                                });

                                            otpTextField.text = qrCode;
                                          },
                                          icon: const Icon(
                                              Icons.camera_alt_rounded),
                                          label: const Text("Scan"))),
                            ),
                          ),
                          Flexible(
                            flex: 1,
                            child: Padding(
                              padding: const EdgeInsets.fromLTRB(10, 0, 5, 0),
                              child: ElevatedButton.icon(
                                  key: const ValueKey("verify_button"),
                                  style: ElevatedButton.styleFrom(
                                      minimumSize:
                                          const Size(double.infinity, 20),
                                      foregroundColor: Colors.white,
                                      backgroundColor: scribePurple,
                                      padding: const EdgeInsets.all(15),
                                      shape: const RoundedRectangleBorder(
                                          borderRadius: BorderRadius.all(
                                              Radius.circular(5)))),
                                  onPressed: () async {
                                    Map<String, dynamic> formValues = _formKey
                                        .currentState!.fields
                                        .map((key, field) =>
                                            MapEntry(key, field.value));

                                    String email = formValues['email']
                                        .toString()
                                        .toUpperCase();

                                    String otp = formValues['otp'].toString();

                                    // [!] Verify email is in proper format

                                    showFullScreenLoader(context: context);

                                    AuthStatus verifyStatus = await AuthService
                                        .authenticateInputToken(
                                      email: email,
                                      token: otp,
                                      useOTP: true,
                                    );

                                    closeLoader(context: context);

                                    if (verifyStatus == AuthStatus.BAD_TOKEN) {
                                      showSnackbar(
                                          context: context,
                                          message: "Invalid OTP code",
                                          type: SnackbarMessageType.BAD);

                                      return;
                                    } else if (verifyStatus ==
                                        AuthStatus.MAX_ATTEMPTS_REACHED) {
                                      showSnackbar(
                                        context: context,
                                        message:
                                            "Max Attempts Reached. Request a New Code or Forgot Password?",
                                        type: SnackbarMessageType.BAD,
                                      );
                                      return;
                                    }

                                    // ignore: use_build_context_synchronously
                                    showSnackbar(
                                        context: context,
                                        message: "OTP accepted!",
                                        type: SnackbarMessageType.GOOD);

                                    print(
                                        "[!] ✅ Good token, navigate to main page now");

                                    await AuthService.initialize();
                                    await RateLimiting.initialize(
                                        context: context);
                                    await FeatureFlag.initialize();

                                    Segment.trackLogin();

                                    if (FeatureFlag.appointments) {
                                      Navigator.pushReplacement(
                                        context,
                                        MaterialPageRoute(
                                            builder: (context) =>
                                                WindowControlsWrapper(
                                                    nestedScreen:
                                                        const AppointmentScreen())),
                                      );
                                    } else {
                                      Navigator.pushReplacement(
                                        context,
                                        MaterialPageRoute(
                                            builder: (context) =>
                                                WindowControlsWrapper(
                                                    nestedScreen:
                                                        RecordingScreen(
                                                  linkedAppt: null,
                                                ))),
                                      );
                                    }
                                  },
                                  icon: const Icon(Icons.verified_user_rounded),
                                  label: const Text("Verify")),
                            ),
                          )
                        ],
                      )
                    : Container(),
                const SizedBox(height: 50),
                TextButton.icon(
                    key: const ValueKey("toggle_login_button"),
                    onPressed: () {
                      setState(() {
                        isLogin = !isLogin;
                      });
                    },
                    icon: const Icon(Icons.login_outlined),
                    label: Text(isLogin
                        ? "First time? Create an account."
                        : "Already have an account? Login")),
                const Spacer(),
                UniversalPlatform.isIOS
                    ? Row(
                        mainAxisAlignment: MainAxisAlignment.spaceBetween,
                        children: [
                          AgreementLinkButton(
                              agreementType: AgreementType.EULA),
                          AgreementLinkButton(
                              agreementType: AgreementType.PRIVACY),
                        ],
                      )
                    : Container()
              ],
            ),
          ),
        ));
  }
}

none of the above worked, even the execute script seems to be having no effect at all in safari iOS.

How can we reproduce the issue?

create basic flutter web app
add FormBuilderTextField with any text

 Flexible(
                      flex: 2,
                      child: FormBuilderTextField(
                        key: const ValueKey("email_input"),
                        name: "email",
                        decoration: const InputDecoration(
                            label: Text("Email"),
                            contentPadding: EdgeInsets.fromLTRB(15, 15, 15, 15),
                            border: OutlineInputBorder(
                                borderRadius:
                                    BorderRadius.all(Radius.circular(5)))),
                      ),
                    ),

run appium selenium XCUITest driver try to fill in some text

### Relevant log output

```shell
No output, seems the event gets sent but nothing happens in the website

Operating System

iOS

Selenium version

4.9.0

What are the browser(s) and version(s) where you see this issue?

Safari

What are the browser driver(s) and version(s) where you see this issue?

Safari 17

Are you using Selenium Grid?

No

[github-actions[bot]]

@jay-athelas, thank you for creating this issue. We will troubleshoot it as soon as we can.

---

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

[diemol]

The right place to fill this issue is the Appium project.

[github-actions[bot]]

This issue has been automatically locked since there has not been any recent activity since it was closed. Please open a new issue for related bugs.