leancodepl / patrol

Flutter-first UI testing framework. Ready for action!
https://patrol.leancode.co
Apache License 2.0
846 stars 125 forks source link

patrol test run tests in alphabetical order again since patrol_cli version 2.4.0 #2147

Open TheSimodon opened 5 months ago

TheSimodon commented 5 months ago

Steps to reproduce

This is a problem was posted around summer last year as a feature request which was implemented a while back.

Reproduction: Have multiple tests run in a specific order.

Example: patrol test ios -d "${DEVICE_ID}" \ --target integration_test/C_test.dart \ --target integration_test/A_test.dart \ --target integration_test/B_test.dart \

Execuion: Tests will be executed in alphabetical order A_test > B_test > C_test

This started to happen since the patrol_cli version 2.4.0, I don't know if the feature was abandoned or it is a bug but for larger projects with hundreds of tests... this feature was quite neat.

Actual results

Tests run in alphabetical order rather than in the designated targeted order

Logs

Logs ``` ```

Patrol version

patrol: 3.5.2

Patrol Doctor output

Patrol Doctor output ``` Patrol CLI version: 2.6.4 Android: • Program adb found in /opt/homebrew/bin/adb • Env var $ANDROID_HOME is not set. (See the link: https://developer.android.com/tools/variables#set) iOS / macOS: • Program xcodebuild found in /usr/bin/xcodebuild • Program ideviceinstaller found in /opt/homebrew/bin/ideviceinstaller ```

Flutter Doctor output

Flutter Doctor output ``` Flutter 3.16.9 • channel stable • https://github.com/flutter/flutter.git Framework • revision 41456452f2 (7 weeks ago) • 2024-01-25 10:06:23 -0800 Engine • revision f40e976bed Tools • Dart 3.2.6 • DevTools 2.28.5 ```
bartekpacia commented 5 months ago

probably related: https://github.com/leancodepl/patrol/issues/1467