MobileNativeFoundation / bluepill

Bluepill is a reliable iOS testing tool that runs UI tests using multiple simulators on a single machine
BSD 2-Clause "Simplified" License
3.19k stars 232 forks source link

When running a specific scheme , Bluepill takes all the tests that including and excluding in this scheme #428

Closed moatazeldebsy closed 1 year ago

moatazeldebsy commented 4 years ago

When running a specific scheme, Bluepill takes all the tests that including and excluding this scheme.

Version: Bluepill-v5.2.0 Xcode version: 11.3

I'm building the app for test using the following command:

xcodebuild \
  -workspace Demo.xcworkspace \
  -scheme DemoUISmokeTests\
  -destination "platform=iOS Simulator,OS=12.2,name=iPhone XS Max" \
  clean build-for-testing

and inside this scheme we have 30 tests

and after that use the xctestrun file in the following bluepill command:

./bluepill --xctestrun-path DemoUISmokeTests_iphonesimulator13.2-x86_64.xctestrun -o test-results -d "iPhone Xs" -H -n 3 -R 1 -f 1 -F true -r "iOS 13.3"

but after running this I noticed that the bluepill running all UI Tests (68) tests not only the scheme.

Also, the parallel run is splitting the scheme or running all the same tests with different devices

Screenshot 2020-04-04 at 15 49 13

Thanks

moatazeldebsy commented 4 years ago

Any updates about this issue

chenxiao0228 commented 3 years ago

@moatazeldebsy Thanks for reporting this issue.

Any chance you could create a sample project that we can use to reproduce this issue and also include in our integration tests?