search

TheCacophonyProject / bird-monitor

Android application to regularly record birdsong for analysis
GNU General Public License v3.0
11 stars 9 forks source link

Improve tests by creating base class and tried to make less flaky and made sure all tests use test server. #55

Closed jackodsteel closed 5 years ago

jackodsteel commented 5 years ago

Updated tests to inherit from a base test class that manages all the standard setup/teardown to reduce code duplication.

Made the tests fully restart/reset on each test case which makes them fully independent, they shouldn't rely on any initial state anymore.

Made all tests use the test server via the base test class.

Updated idlingResources to be incremented/decremented in the Server class mostly, to prevent issues with message handling. This has the downside that there might be slight gaps between idlingResources (e.g. sign in -> get groups). Doesn't seem to make the tests fail, but may be an issue if we want to remove sleeps.

Renamed a couple of things to be more clear/specific (disable -> disableAutomaticRecording; settingsForTestServerEnabled -> veryAdvancedSettingsEnabled)

Added extra library to allow "scroll" instead of "swipe" which is far more consistent for navigation.

Made preferences fully save/restore between each test so running tests on your real device leaves its settings consistent with how it was before.

jackodsteel commented 5 years ago

@timhot I was able to have the recordAndSaveOnServer fail consistently, I think it should be fixed now but if you can double check. I've also added the change to always use the test server as requested :)

timhot commented 5 years ago

Great work. Instead of using the timhot account, please create a new account on the test server for testing.

timhot commented 5 years ago

This didn't have conflicts before I merged #54 I'll leave for you to sort :-)