Expose hot restart to flutter_driver

[ericwindmill]

Use case

Writing e2e tests could be more automated, and less time consuming, if we could wipe app state at the end of any given test. Often, we don't want the state from the previous test to carry over to a new test or workflow that we're testing.

Proposal

Expose a method on FlutterDriver that performs a hot restart on the app that's running for the driver. (i.e. FlutterDriver.restartApp), or add a flag to the driver that can configure it to restart at the end of every test. (i.e.FlutterDriver.restartAfterEachTest = true`).

more context

My team is re-writting our company's app in Flutter from iOS. The QAEs are attempting to port the E2E tests from our current app with Flutter Driver. Currently, they're written using cucumber.

There actually is a cucumber port on Pub that offers this feature. It accomplishes this by writing an "R" to stdin to the vm running the app while driver is running. That library isn't great for our use case, I'm just pointing it out for context

[nerder]

This is very interesting and useful, at the moment it should be doable using requestData in this way:

await driver.requestData("hot_reload")

  enableFlutterDriverExtension(handler: (message) async {
    if(message == "hot_reload") {
      Process.killPid(PID_OF_YOUR_FLUTTER_RUN_PROCESS, ProcessSignal.sigusr2);
    }
  });

But this will add additional flakiness to the tests since as far as I know there is no way to understand when the hot reload process is done, so it might be possible that you start the hot reload and the test kicks in too soon causing the test to fail since the app is not ready yet.

Also, in trying this one out I got the driver to crash with this error VMServiceFlutterDriver: Instance of '_WebSocketImpl' is closed with an unexpected code 1005 so I think it's probably possible to be implemented as a custom behavior but requires a bit more investigations.

Hopefully, somebody can point me in the right direction here.

Edit: Another important missing to make this happen is to be able to save a pid file using --pid-file in flutter drive. An issue is open here: https://github.com/flutter/flutter/issues/54004

[johnpryan]

cc: @jonahwilliams

[deathcoder]

actually, you cannot send that kill signal from the driver as it runs in the app, and the process runs on your pc. What you could do is send that signal from your tests, we tried it but an hot restarts causes the driver connection to drop as @nerder mentioned, at the moment we have a script that launches e2e tests and forces hot restart before launching the driver test. you need a separate test file for each time you need to hot restart but at least it works.

probably if you have a setup method that closes the driver connection, hot restarts and then reopens the driver connection than you would be able to have all of the hot restart logic inside your tests... will update with a solution if this works

[orestesgaolin]

This is how I achieve hot-restart in my flutter driver tests:

test_app.dart:

import 'package:flutter_driver/driver_extension.dart';
import 'package:gemini/main/main_mock.dart' as app;
import 'package:hydrated_bloc/hydrated_bloc.dart';

void main() {
  enableFlutterDriverExtension(handler: (command) async {
    switch (command) {
      case 'restart':
        await HydratedBloc.storage.clear(); // clearing cache between each hot-restart
        _startApp();
        return 'ok';
    }
    throw Exception('Unknown command');
  });
  _startApp();
}

void _startApp() {
  app.main(); // this is call to runApp(...)
}

in my tests in setUp before each test:

    setUp(() async {
      await driver.restart();
    });

and the extension:

extension DriverExt on FlutterDriver {
  Future<void> restart() async {
    await requestData('restart');
  }
}

[lhimo]

Is there a way to do this within the new integration_test package?

[johnpryan]

@lhimo are you having this issue with integration_test? Since integration_test tests are written using flutter_test, the app state should be re-created whenever pumpWidget() is invoked.