search

sglanzer-deprecated / ember-cli-blanket

Blanket based code coverage for ember-cli projects
MIT License
110 stars 46 forks source link

Any way to run code coverage via command line? #13

Closed achan closed 9 years ago

achan commented 9 years ago

I'm new to ember-cli and Ember in general, but is there a way to run blanket from command line? Any options to enable it via ember test? Moreover, is there an easy way to generate lcov reports to feed in to services like coveralls / code climate?

sglanzer-deprecated commented 9 years ago

@achan The blanket documentation is a little scattered https://github.com/alex-seville/blanket#blanketjs, but it supports custom reporters, so the ability to generate lcov reports is definitely a possibility. I've also seen some examples (I can't remember where) that demonstrated running headless with PhantomJS, so running via ember test should also be a possibility.

Currently the integration project doesn't support either of these. I may find time to investigate this eventually, but the current level of coverage reporting meets our project's needs, so it's taking a back seat to other framework projects. If you're interested in investigating either or both of these issues let me know and I'd certainly be happy to give you a hand in finding the hooks/explaining the current integration setup. I do intend on enhancing this project in the future, but it's definitely going to move faster if other people want to get involved.

achan commented 9 years ago

I've forked the repo and I'll see how much work it would take to get this running headless first...

kiwiupover commented 9 years ago

thanks @achan for looking into command line interface I would totally use it. cheers

sglanzer-deprecated commented 9 years ago

We're going to start looking into the headless run as well as LCOV reports that can be fed into systems like Sonar or TeamCity. @achan Have you made any progress looking into this that we could build on?

achan commented 9 years ago

Nope sorry, not able to work on this at the moment..

sglanzer-deprecated commented 9 years ago

@achan No problem, just wanted to make sure we didn't duplicate effort.

kiwiupover commented 9 years ago

@achan Let me know if I can help with the headless coverage reports either testing or coding.

achan commented 9 years ago

@kiwiupover I'm pretty much not involved with this issue other than creating it.. I believe @sglanzer is your guy :)

kiwiupover commented 9 years ago

Ok I get it.

grese commented 9 years ago

@sglanzer - I would like to get involved.. Our team really needs headless tests, and custom coverage reporting. I have some spare time to spend on this because it is the only item keeping our team from onboarding to ember-cli :)

Could you please point me to the necessary hooks as you mentioned above? I would love these features :)

sglanzer-deprecated commented 9 years ago

@grese Excellent, getting this feature would be great.

https://github.com/alex-seville/blanket/tree/master/test/custom-reporter is the basic blanketjs example for custom reporters.

This appears to be reasonable example for running with phantomjs and a custom reporter: https://github.com/ModelN/grunt-blanket-qunit

That uses grunt but I would imagine that it could also be driven using testem (standard ember-cli).

The current implementation takes the result of the ES6 to ES5 conversion performed in the ember-cli build, loops over the entries store in the requirejs.entries list, instruments each entry using blanket, loads all of the entries and then lets qunit start the test run (there is a flag set to block the run until the entries have been loaded).

The mocha case is a bit different, @mheisig did the work for mocha, so if you want to go that route he would be able to give you more information.

I know this isn't a lot to go on, but let me know if I can do anything else to help.

sglanzer-deprecated commented 9 years ago

Another note - I'm not a fan of how the project hooks into ember-cli currently, it was a hack effort and I think modifying the instrumentation to run during the postProcess hook http://hashrocket.com/blog/posts/a-compendium-of-hooks-in-embercli#postprocess would be a better idea.

If you can get the headless runner going with the current implementation, cool - otherwise we may need a bit better control over the execution and I think that hook might be the right way to get better control.

grese commented 9 years ago

I've managed to get the lcov file working when the user hits localhost:4200?coverage=true (with both mocha and qunit), but still need to get it working in the headless runner...(that's where the bulk of the work will be, I suppose.).

Thanks for the pointers :) I'll also look into the hooks while I'm at it because I think your're correct - (we will probably need more control to make this work in headless runners).

max-winderbaum commented 9 years ago

@grese, could you publish your work thusfar? I'd love to see it :+1:

DebVortex commented 9 years ago

It has been a couple of days, since @grese posted that he working on a approach of tackling headless coverage.

Does anyone have new information about that? I'm really interested in this topic as well.

jschilli commented 9 years ago

I dug into this a few days ago while looking @ the overall coverage updates

Stating the obvious we need to: [] generate an lcov report from blanket [] get the lcov report written to the filesystem

I'm working on a POC using an ember test middleware to take the results when the the blanket test ends.

setting up the lcov reporter is a bit of PITA but I'm close - will advise