gotwarlost / istanbul

Yet another JS code coverage tool that computes statement, line, function and branch coverage with module loader hooks to transparently add coverage when running tests. Supports all JS coverage use cases including unit tests, server side functional tests and browser tests. Built for scale.
Other
8.7k stars 786 forks source link

Coverage artifacts in js code when stringifying a function at runtime. #674

Open bsommardahl opened 8 years ago

bsommardahl commented 8 years ago

I learned about how Istanbul measures coverage in a really cool way thanks to my current problem. My current task is to write a javascript function and then return the function as a string. My unit test attempts to eval the string function and fails because of the following error:

ReferenceError: __cov_HnbYpbOoQM4MaLYAY_l6Ig is not defined

When I take a look at the value of my function string, I see this:

function (browser,tabTitle){
    __cov_HnbYpbOoQM4MaLYAY_l6Ig.f['3']++;
    __cov_HnbYpbOoQM4MaLYAY_l6Ig.s['10']++;
    var currentTabId=browser.getCurrentTabId();
    __cov_HnbYpbOoQM4MaLYAY_l6Ig.s['11']++;
    browser.getTabIds().forEach(id=>{
        __cov_HnbYpbOoQM4MaLYAY_l6Ig.s['12']++;
        browser.switchTab(id);
        __cov_HnbYpbOoQM4MaLYAY_l6Ig.s['13']++;
        if(browser.getTitle().indexOf(tabTitle)>-1){
            __cov_HnbYpbOoQM4MaLYAY_l6Ig.b['2'][0]++;
            __cov_HnbYpbOoQM4MaLYAY_l6Ig.s['14']++;
            return false;
        }
        else {
            __cov_HnbYpbOoQM4MaLYAY_l6Ig.b['2'][1]++;
            __cov_HnbYpbOoQM4MaLYAY_l6Ig.s['15']++;
            browser.switchTab(currentTabId);
        }
    });
}

Cool huh? Now I understand how instanbul is able to see what code was visited during unit test execution. In the coverage context, I'm sure __cov_HnbYpbOoQM4MaLYAY_l6Ig is defined and works fine. But, in my unit test context, it doesn't exist. :)

Here are a couple of gulp tasks that I think are relevant:

gulp.task('pre-test', function () {
  return gulp.src(['build/**/*.js', '!build/src/features/**/*'])
    .pipe(istanbul({includeUntested: true}))
    .pipe(istanbul.hookRequire());
});
gulp.task('unit-tests',** ['clean-coverage', 'pre-test'], function(){
    return gulp.src(['build/src/**/*.spec.js'])
        .pipe(mocha({reporter: 'spec'}))
        .pipe(istanbul.writeReports({
            dir: './coverage',
            reportOpts: {
                dir: './coverage'
            },
            reporters: ['text-summary', 'html']
        }))
        .pipe(istanbul.enforceThresholds({ thresholds: { global: 80 } }));
});

So, I think I need a way to prevent istanbul from adding coverage code to this function. I have tried /* istanbul ignore next */ in various places with no luck (coverage code still present). I would gladly provide my unit test context with a defined __cov_HnbYpbOoQM4MaLYAY_l6Ig but that slug changes each time.

Not sure what to try next. Any help would be appreciated!

tagyoureit commented 7 years ago

This (https://github.com/gotwarlost/istanbul/issues/310) helped me resolve the issue at runtime. Maybe it will help you with Gulp tasks, too?