lyndsey-ferguson / fastlane-plugin-test_center

🎯 The best fastlane plugin to understand and tame misbehaving iOS tests 🎉
MIT License
285 stars 66 forks source link
fastlane fragile-tests ios junit mac testing testing-tool xcode xcuitest

test_center plugin 🎯

fastlane Plugin Badge Actions Status

Have you ever spent too much time trying to fix fragile tests only to give up with nothing real to show? Use the fastlane actions from test_center to remove the pain around your tests, so that you can focus on what makes 💰: features that customers love 😍.

For those of you new to fastlane, I recommend that you read my article Rescue Your Mobile Builds from Madness Using Fastlane.

Features | Installation | Usage | Issues & Feedback | Contributing | Troubleshooting | License

Features

This plugin makes testing your iOS app easier by providing you actions that give you greater control over everything related to testing your app.

multi_scan began when I created an action to only re-run the failed tests in order to determine if they were truly failing, or if they were failing randomly due to a fragile infrastructure. This action morphed into an entire plugin with many actions in the testing category.

This fastlane plugin includes the following actions:

read the documentation on each action by clicking on the action name

Action Description Supported Platforms
multi_scan supports everthing that scan (also known as run_tests) does, and also supports:

- dividing your tests evenly into batches and run each batch on its own Simulator in parallel to reduce the time to test
- re-running tests that may be failing due to a fragile test environment
- splitting tests into batches when a huge number of tests overwhelm the Simulator
- performing an action after a block of tests have been run
ios, mac
suppress_tests_from_junit suppress tests in an Xcode Scheme using those in a Junit test report ios, mac
suppress_tests suppress tests in an Xcode Scheme ios, mac
suppressed_tests returns a list of the suppressed tests in your Xcode Project or Scheme ios, mac
test_options_from_testplan returns the tests and test code coverage configuration for a given testplan ios, mac
testplans_from_scheme returns the testplans that an Xcode Scheme references ios, mac
tests_from_junit returns the passing and failing tests in a Junit test report ios, mac
tests_from_xcresult returns the passing and failing tests in a xcresult bundle ios, mac
tests_from_xctestrun returns a list of tests for each test target in a xctestrun file ios, mac
collate_junit_reports combines multiple Junit test reports into one report ios, mac
collate_html_reports combines multiple HTML test reports into one report ios, mac
collate_json_reports combines multiple json test reports into one report ios, mac
collate_test_result_bundles combines multiple test_result bundles into one test_result bundle ios, mac
collate_xcresults combines multiple xcresult bundles into one xcresult bundle ios, mac

Installation

This project is a fastlane plugin. To get started with fastlane-plugin-test_center, add it to your project by running:

fastlane add_plugin test_center

Usage

Click the name of each action above and review the documentation to learn how to use each action.

The most popular action in the test_center plugin is multi_scan, and if you run your tests in parallel with multiple retries, they will finish faster and only the truly failing tests will be reported as failing:

multi_scan(
  project: File.absolute_path('../AtomicBoy/AtomicBoy.xcodeproj'),
  scheme: 'AtomicBoy',
  try_count: 3, # retry _failing_ tests up to three times^1.
  fail_build: false,
  parallel_testrun_count: 4 # run subsets of your tests on parallel simulators^2
)
# [1] The ones that pass on a retry probably failed due to test interactions or test infrastructure
# [2] splits all your tests into 4 smaller batches and runs each batch on its own sim in parallel for faster completion!

Issues and Feedback

For any other issues and feedback about this plugin, please submit it to this repository.

Contributing

If you would like to contribute to this plugin, please review the contributing document.

Troubleshooting

If you have trouble using plugins, check out the Plugins Troubleshooting guide.

Using fastlane Plugins

For more information about how the fastlane plugin system works, check out the Plugins documentation.

About fastlane

fastlane is the easiest way to automate beta deployments and releases for your iOS and Android apps. To learn more, check out fastlane.tools.

Supporters

Владислав Давыдов vdavydovHH

License

MIT