jejacks0n / teaspoon

Teaspoon: Javascript test runner for Rails. Use Selenium, BrowserStack, or PhantomJS.
1.43k stars 243 forks source link

Proposal - separate coverage configs for browser and CLI runs #379

Closed mkcode closed 9 years ago

mkcode commented 9 years ago

Hi All, I've been using teaspoon for a few months now, and it's been great, especially with all the recent updates. Thanks for a great gem!

Ran into a small issue last week where I was getting istanbul coverage reporting while running teaspoon in the browser. Running Istanbul in the browser: A) makes the test suite slower ( double the files to load ) and B) near impossible to use chrome's debugger ( it references the istanbul wrapped code ). Turns out that this occurred because the app is using dotenv (https://github.com/bkeepers/dotenv) and set COVERAGE=DEFAULT in the app's .env file.

I find that for my own use, I always want a separate configuration when running the specs in the browser ( /teaspoon ) as opposed to the CLI ( via rake, guard, and on CI server ). At least, I would never want coverage generated from the browser runs.

Opening this issue because I suspect that other users would think similarly. Would you consider creating 2 default coverage configuration paths? One for browser runs and one for CLI runs? Short of that, not respecting code coverage ENV variables in browser runs?

Having these 2 default configuration paths would simplify configuring teaspoon considerably. By 2 default configuration paths, I mean, the generated template would create two intuitively named (browser, cli)? coverage configs and teaspoon would be setup by default to use its run context respective config.

jejacks0n commented 9 years ago

What I usually do is setup a different suite that inherits from the default (let's call that one CI). Then the standard CLI behaves like the browser so there's no surprises there, and if you specify the suite as CI via the command line you get the reporting as desired. There's plans to make this easier to accomplish via the rake task interface, but until that happens this is the best solution.

That's my official recommendation. FWIW, I don't think your idea is flawed or anything, I just want to avoid confusion where possible.