RocketChat / Rocket.Chat.iOS

Legacy mobile Rocket.Chat client in Swift for iOS
https://rocket.chat
MIT License
1.03k stars 438 forks source link

[TESTS] Implement UI tests #798

Open macabeus opened 6 years ago

macabeus commented 6 years ago

I started to add UI tests in pull request #743 but I noticed that we have some works to do previously. And I have a few questions:

In first and second topics, one alternative is to add the command line argument, so that the developer can choose the server and account to run the UI tests but I don't know if it's really necessary. That is, use by default demo.rocket.chat and a default account to run tests, and, if developer sets a parameter, use the server/account set by the developer.

If anybody has suggestions or other topics to be add feel free to give any kind of feedback please.

rafaelks commented 6 years ago

You're right @macabeus! We really need to write UI tests and your requirements are correct. Let me try to answer to your questions:

  1. No. We won't use any server to run our tests. There's a requirement we can't have to run the tests, they need to be able to run with no internet connection or server online. This kind of tests is going to be done in the future, to make sure our app is working with different version of the server (even new ones). Right now, the UI tests needs to be done using mocks of requests & results;
  2. I believe the 1 answers this question. Let's simulate with a correct user and an incorrect user;
  3. Maybe we can extend the class and override it's method? We need to get some thoughts on that one; 🤔
  4. Again, test with a success and failure result;

Let's remember the idea of our UI tests: test only the UI. We should not try to test the REST/Socket response from the server in this situation. That's why mocks are more interesting for us. The UI needs to work for all the situations.

rafaelks commented 6 years ago

Looks like Starscream now supports mocks to test, I'm updating the library in a few, then we can start looking into it: https://github.com/daltoniam/Starscream/pull/311.

artur-ios-dev commented 6 years ago

Do we want to use Xcode's UI Tests or maybe try something like EarlGrey which allows us to test UI with unit tests?

Anyway, either way it is we will need some kind of mechanism that will allow us to open certain controller right away (skipping .auth or .chat controller at the start) otherwise tests would take way too long. Also it'll increase the PR's checking by a lot :(

Anyway looking forward for that because without UI tests it's really hard to not decrease total tests coverage.

rafaelks commented 6 years ago

@artrmz I know that Xcode's UI tests are very hard to be done, specially if you've network requests involved. I would say that using a tool like EarlGrey would resolve our problems. Have you used it already?

artur-ios-dev commented 6 years ago

@rafaelks Used that a little bit at work, it’s not perfect but worsk quite well. Would have to write some wrapper around EarlGray in case we decide to use something else in the future and make it easier/faster to use. Can take a look at it when I have time but would need some halp/tips how to mock communication.

rafaelks commented 6 years ago

@artrmz Sure thing, that would be amazing! Let me know when you want to start working on it. 👍