Making test suite run faster

[cojoj]

At the moment of writing Marathon's test suite contains of 51 tests and it takes forever to run them (I know, I can buy iMac Pro... 🙄). It's really not cool to run them locally to check for regression as it takes a lot of time, so we leave regression to CI only 😂

I'd love to see Marathon's test suite run faster, so we can benefit from running tests often and encourage new contributors to write tests and maybe follow TDD.

I have a gut feeling that improving this can also fix #98 😜

This issue is rather a discussion ( @JohnSundell please add label ) as I don't have a clear vision on how to improve the speed. One thing which is obvious is that we don't have mocks at all, so in every test case, se run everything from scratch and it may brings some problems.

[JohnSundell]

@cojoj This has already been discussed here: https://github.com/JohnSundell/Marathon/issues/35 🙂 Since we started running the tests in parallel we felt that they were running fast enough. Sure, they could still run faster, but since we do run everything with the real file system, performing real clone operations etc, things are always going to be a bit slow. It's a tradeoff worth making for this project IMO 🙂

[cojoj]

I had one idea, but I guess with --parallel it doesn't make any sense. I see benefits of doing real operations (not mocking), and for this project I wouldn't change it, but I was thinking about reducing some actions by grouping together tests which can form some sort of scenario, but since --parallel... ¯_(ツ)_/¯

---

BTW, @JohnSundell, if we're on testing... What's the point of having .marathonTests with UUID subfolders? I was also thinking about cleaning up after completing test suite and removing .marathonTests.

[JohnSundell]

@cojoj The UUID subfolders are used to enable parallelization, to make sure all the test cases operate in a unique folder. All for cleaning up the .marathonTests folder, how are you thinking about doing that? Because it can only be cleaned once the entire suite has finished.

[cojoj]

We can clean up in every tearDown since we're recreating everything in setUp, right? There's also one nice thing here https://stackoverflow.com/a/29822900/1911042

[JohnSundell]

@cojoj We can't clean up the main Marathon-folder (~/.marathonTests) in a tearDown since the tests are parallelized 🙂

[cojoj]

Yeah, ok, but take a look at override class func tearDown() - it's called after executing all tests.

[JohnSundell]

Oooh! I've always thought that tearDown gets called after each test method 😮 Well, you live and you learn 😄 Then we should be able to use tearDown, like you say. Feel free to submit a PR 👍