VeryGoodOpenSource / very_good_test_runner

A test runner for Flutter and Dart created by Very Good Ventures 🦄
https://pub.dev/packages/very_good_test_runner
MIT License
18 stars 3 forks source link

fix: `flutter` command gets locked when trying to run sharded tests parallely #25

Open RishabJaiswal opened 4 months ago

RishabJaiswal commented 4 months ago

Description When running multiple flutter test commands parallely via flutterTest() method for sharded tests, the cli throws the below error

Waiting for another flutter command to release the startup lock...

If the same flutter test command is ran for sharded tests without using flutterTest() method but via Process.start, the flutter command runs the sharded tests successfully.

Steps To Reproduce

  1. Call the below methods simultaneously
    
    flutterTest(
          arguments:
          "--total-shards 2 --shard-index 0 '${testDirectory.path}/example_a[1]_test.dart'"
              .split(' '),
          workingDirectory: tempDirectory.path,
        );

flutterTest( arguments: "--total-shards 2 --shard-index 1 '${testDirectory.path}/example_a[2]_test.dart'" .split(' '), workingDirectory: tempDirectory.path, );


**Expected Behavior**
When executing the `flutter test` command via `Process.start` for the shared tests, it runs without failure. 
`flutterTest()` method also uses `Process.start` internally & it should run without locking `flutter` command on the sharded tests.

```dart
Process.start(
    'flutter',
    "--total-shards 2 --shard-index 1 '${testDirectory.path}/example_a[1]_test.dart'"
              .split(' '),
  );
Process.start(
    'flutter',
    "--total-shards 2 --shard-index 2 '${testDirectory.path}/example_a[2]_test.dart'"
              .split(' '),
  );

Screenshots

image

Additional Context Please find the test case to reproduce the issue here:

test('emits correctly for sharded tests (e2e)', () async {
      final tempDirectory = Directory.systemTemp.createTempSync();
      File('${tempDirectory.path}/pubspec.yaml').writeAsStringSync(
        '''
name: example
version: 0.1.0+1

environment:
  sdk: ">=2.12.0 <3.0.0"

dev_dependencies:
  test: any
''',
      );
      final testDirectory = Directory('${tempDirectory.path}/test')
        ..createSync();
      File('${testDirectory.path}/example_a[1]_test.dart').writeAsStringSync(
        '''
import 'package:test/test.dart';

void main() {
  test('example', () {
    expect(true, isTrue);
  });
}
''',
      );

      File('${testDirectory.path}/example_a[2]_test.dart').writeAsStringSync(
        '''
import 'package:test/test.dart';

void main() {
  test('example', () {
    expect(true, isTrue);
  });
}
''',
      );
      expect(
        flutterTest(
          arguments:
              "--total-shards 2 --shard-index 0 '${testDirectory.path}/example_a[1]_test.dart'"
                  .split(' '),
          workingDirectory: tempDirectory.path,
        ).where((e) => e is DoneTestEvent).first,
        completes,
      );

      expect(
        flutterTest(
          arguments:
          "--total-shards 2 --shard-index 1 '${testDirectory.path}/example_a[2]_test.dart'"
              .split(' '),
          workingDirectory: tempDirectory.path,
        ).where((e) => e is DoneTestEvent).first,
        completes,
      );
    });