Build an efficient process for testing the app

[brainwane]

This is a TODO from https://zulip.readthedocs.io/en/latest/roadmap.html that Tim included in the Zulip roadmap when he first published it in 27b8e8b294e3c43566dea724d059ef32af16a8f3 .

[akashnimare]

@brainwane Sorry for the late reply as I was on a little vacation 😄 . I have not done much work on testing but releasing new versions, auto updates etc will be covered in our first stable release.

[geeeeeeeeek]

What kind of tests are you looking for? Since our app is directly fetched from the web, there is only few things we can test on Electron, like testing if each menu item works.

[timabbott]

Yeah I think the things I'd have automated tests for are:

• Areas we've had a lot of bugs, e.g. the handling of the server URL entered on the first screen would be a good candidate for this
• That the various menu items work
• Basic testing of any dependencies we've had trouble with.

And then maybe part 2 of this (which probably deserves its own issue) is that we should have a beta channel in the auto-update system, so that we can push an update to only some people, wait a few days for issue reports, and then push it out to the rest.

[akashnimare]

Basic testing of any dependencies we've had trouble with.

@timabbott specially Spellchecker.

[geeeeeeeeek]

I'll try to write a plan. And see what we can do on testing.

[priyank-p]

@timabbott i can start working on test using mocha if you like.

[akashnimare]

For testing, have a look test-spellchecker branch (it's WIP). What kind of test you want to work on? @SimplyAhmazing have added couple of e2e tests here.

[priyank-p]

I am working with Ahmed Abdalla on that one edit: with @SimplyAhmazing.

[SimplyAhmazing]

@cPhost There 3 categories of testing for an electron application - testing code in the renderer process, testing code in the main process, and integration tests that runs the entire app with both processes live.

• renderer process tests - I just submitted this PR for writing tests that are executed in the renderer process
• main process - we don't have any tests/setup for running main process tests. main process apply to things like autoupdater or other modules that need an instance app singleton and other electron resources. We can go about doing this by mocking electron imports such as referenced here. Also came across using mocha and passing it the electron binary. I think the simplest thing to do is make main process packages require electron resources as dependencies and mock the deps (dependency injection, factory pattern).
• integration tests - integration tests start the who application and allow you to interact with the application using Spectron and webdriverio

After this PR is closed the renderer process tests are in tests/unit and integration tests are in tests/e2e. Nothing is setup yet for main process tests.

/cc @akashnimare