search

duckduckgo / iOS

DuckDuckGo iOS Application
https://itunes.apple.com/us/app/duckduckgo-privacy-browser/id663592361?mt=8
Apache License 2.0
1.82k stars 414 forks source link

scripts for running UI tests #3000

Closed brindy closed 3 months ago

brindy commented 3 months ago

Task/Issue URL: https://app.asana.com/0/392891325557410/1207589761442405/f Tech Design URL: CC: @bwaresiak

Description: Adds scripts for running the UI tests locally in a way that's similar to how they run on CI.

Setting up UI tests:

  1. Run .maestro/setup_ui_tests.sh from the .maestro folder. It should tell you to run it from the root. If there are other errors with your environment (such as missing commands) it will also fail.
  2. Run .maestro/setup_ui_tests.sh from the root. It'll build the app to a local DerivedData folder, create a simulator and install the app
  3. Run .maestro/setup_ui_tests.sh --skip-build. It'll skip the build, but create a simulator and install the app if one is available in the right location

Running UI tests:

  1. Run .maestro/run_ui_tests.sh. It'll fail because you need to specific a location for the flow files to run.
  2. Run .maestro/run_ui_tests.sh --flow-location .maestro\x which will fail because the location is wrong.
  3. Run .maestro/run_ui_tests.sh --flow-location .maestro\release_tests\ it will run the tests and create a file called run_log.txt in DerivedData. Once it finishes it'll dump that file to the console. If they all passed it will show a success message.
  4. Run .maestro/run_ui_tests.sh --run-flow it'll fail because you need to specify a flow file
  5. Run .maestro/run_ui_tests.sh --run-flow .maestro/release_tests/bookmarks.yaml it'll run just that test
  6. Run .maestro/run_ui_tests.sh --flow-location .maestro\release_tests\ and hit Ctrl-C. This will make it appear like the test has failed. Do this for a few test and allow others to pass. The final message should indicate that some failures occurred.

Cleaning up:

  1. Run .maestro/clean_simulators.sh and it'll kill the simulators and then remove the ones that have 'maestro' in the name. Some errors might get printed for where they don't need to be shutdown but this can be ignored.
brindy commented 3 months ago

@loremattei addressed your comments - ready for another look, thanks.

brindy commented 3 months ago

@loremattei

Just pushed a few changes