[README] Write about limitation that test files should be run file by file - Githubissues

Genki-S / ttnt

Test This, Not That! (Google Summer of Code 2015 project under Ruby on Rails)

MIT License

263 stars 7 forks source link

[README] Write about limitation that test files should be run file by file #2

Closed Genki-S closed 9 years ago

Genki-S commented 9 years ago

Due to ruby coverage library's limitation, to produce code-to-test mapping one should run tests file by file. Writing the detail about this might be good to get ideas from others.

Genki-S commented 9 years ago

Current limitation:

I can only store coverage information for each test file, not test case
I should run each separate test files in it's own process (all I can do is to wrap the ruby process with Coverage.start and Coverage.result)

This is caused by these limitation of Ruby coverage library:

All the files you want to track coverage should be loaded after Coverage.start or coverage won't be tracked at all (meaning you cannot do Coverage.start multiple times effectively, e.g. in test setup)
When Coverage.result is called, all the coverage results stored are lost (i.e. calling Coverage.result twice, second call returns nothing)