Provide a way for Robolectric to fake objects

mcomella commented 5 years ago

Vision statement / What / Requirements

On-device tests are take a long time to write, take a long time to run, and can be hard to maintain. For these reasons, sometimes it's preferred to write tests that run off-device. After https://github.com/mozilla-mobile/firefox-tv/issues/2367, we're now able to write these tests. However, we're unable to fake some objects during testing like the functionality contained in the ServiceLocator.

Like https://github.com/mozilla-mobile/firefox-tv/issues/1651 did for Espresso, we should figure out how to fake our objects. It looks like we may be able to use similar (the same?) techniques.

Impact

When writing automated tests, we're forced to write some tests as on-device tests (see above for downsides). Fixing this bug will let us write most tests as off-device tests, which gives us a lot more flexibility as developers.

To be fair, we infrequently need these style of tests in the past. However, this may be a chicken and egg problem: we don't know how to quickly & comfortably write these tests so we don't write them. If this style of testing was quickly accessible, we could have used it in #2325 instead of exposing functionality to QA to add it to their test cases (which will take up a lot of time in the long run).

Acceptance criteria

[ ] Developers can provide fake/mock objects into MainActivity during Robolectric testing
- Example test: when MainActivity.onStart is called then verify pocketRepo.updateFromStore is called

mcomella commented 5 years ago

Current alternatives:

Provide mutable values that devs can override at run time
Extract the functionality out of MainActivity and test that new class (though no one tests that MainActivity correctly calls that new class)

athomasmoz commented 5 years ago

Would this decrease our current CI run times? If not, could this be a lower priority?

mcomella commented 5 years ago

Would this decrease our current CI run times?

No – this is primarily about enabling us to easily test a wider variety of code paths.

If not, could this be a lower priority?

We could put it off until someone needs it to test something they want to test: then we can have a discussion.

mozilla-mobile / firefox-tv