istanbuljs / babel-plugin-istanbul

A babel plugin that adds istanbul instrumentation to ES6 code
BSD 3-Clause "New" or "Revised" License
630 stars 75 forks source link

Error when using jest + collectCoverage/babel-plugin-lodash #116

Open namjul opened 7 years ago

namjul commented 7 years ago

I posted this issue before on https://github.com/facebook/jest/issues/3959 and https://github.com/lodash/babel-plugin-lodash/issues/179

Using babel-plugin-lodash, collectCoverage enabled and a lodash function as show below imported from test file, results into an error.

import { flow, flatten } from 'lodash/fp'

export const testFunction1 = flatten 
// export const testFunction2 = flow(flatten) 

example#testFunction1 npm test will output

    /home/nam/projects/jest-babel-lodash-issue/src/example.js:3
    var testFunction1 = /* istanbul ignore next */exports.testFunction1 = ();
                                                                           ^

    SyntaxError: Unexpected token )

example#testFunction2 npm test will output

    ReferenceError: /home/nam/projects/jest-babel-lodash-issue/src/example.js: Container is falsy

      at NodePath._replaceWith (node_modules/babel-traverse/lib/path/replacement.js:170:11)

Removing collectCoverage from package.json lets the tests run without failure. Removing babel-plugin-lodash from .babelrc or importing directly from lodash also runs without failure.

Testcase https://github.com/namjul/jest-babel-lodash-issue

AndersDJohnson commented 7 years ago

Have you tried moving thebabel-plugin-lodash plugin to a non-test environment config?

{
  "plugins": [],
  "env": {
    "production": {
      "plugins": ["lodash"]
     }
  }
}

then when running tests, use:

BABEL_ENV=test jest --coverage

https://github.com/namjul/jest-babel-lodash-issue/issues/1 https://github.com/facebook/jest/issues/3959#issuecomment-317018979

bcoe commented 7 years ago

@namjul feels like an order of operation issue related to the order that the plugins are applied; I've seen similar issues with a few other plugins in the past (babel-plugin-rewire comes to mind).

It might be worth also opening a tracking issue on babel/babel; we can certainly do a bit of work on our our side of things to see if there's a workaround as well. The underlying issue will most likely be in this repo.

namjul commented 7 years ago

@AndersDJohnson thats a workable solution thanks. There is probably still a bug somewhere like @bcoe mentioned.

stephanschubert commented 7 years ago

Hi, I've been searching the issue trackers of jest and others for a while now and this one seems related or very similar to the issue I have right now. That's why I'm not opening a new one.

I'm auto-importing all exported functions fromlodash/fp with babel-plugin-auto-import:

/* eslint-disable */
const candidates = [
    'recompose',
    'lodash/fp',
].map(path => [path, require(path)]);

const toDeclaration = ([path, lib]) => ({ path, members: Object.keys(lib) });

module.exports = {
    plugins: [
        [
            'auto-import', {
                declarations: candidates.map(toDeclaration),
            },
        ],
    ],
};

and import it as preset in my .babelrc:

    "presets": [
        "react",
        ["env", {
            "modules": false,
            "targets": {
                "browsers": ["last 2 versions", "safari >= 7"]
            }
        }],
        "./babel-preset-auto-import.js"
    ],

This works fine until I run jest --coverage and have some unused code, e.g.:

import toArray from './toArray';

const foo = identity; // <- UNUSED

const flatArray = flow(toArray, flatten);
export default flatArray;

Result:

$(npm bin)/jest --coverage test/utils/flatArrayTest.js
 FAIL  test/utils/flatArrayTest.js
  ● Test suite failed to run

    ReferenceError: identity is not defined

      at Object.<anonymous> (src/utils/flatArray.js:3:61)
      at Object.<anonymous> (test/utils/flatArrayTest.js:2:18)
      at process._tickCallback (internal/process/next_tick.js:109:7)

But running npm run build and checking the final code is fine:

import { identity, flow, flatten } from 'lodash/fp';
import toArray from './toArray';

var foo = identity;

var flatArray = flow(toArray, flatten);
export default flatArray;

So it looks to me that the import is not generated for whatever reason when the code's unused during the coverage-generating test run. Maybe some kind of intermediate state of "babelified" code is used?

Running jest without --coverage works as expected.

BobDeKort commented 7 years ago

Having the same issue running without coverage works but I would like a solid solution. Has anyone found anything?

cmswalker commented 6 years ago

It won't help everyone as there are different test pipelines, but for some odd reason i've found that using npx with npx jest --coverage will work in favor of npm scripts and even calling the jest command directly

bcoe commented 6 years ago

@cmswalker @BobDeKort is it the lodash plugin giving you both issues as well? Seems like a frequent enough cause of problems that it might be worthwhile trying to get a fix in for this specific incompatibility.

cmswalker commented 6 years ago

@bcoe yes. We had to pull istanbul out of our deploy pipleline and stick to using it on the side. Not sure why/how there is an inconsistency with ./node_modules/.bin/jest --coverage vs npx jest --coveragebut there is at the moment.

bcoe commented 6 years ago

@cmswalker @BobDeKort labeled this as high priority, to remind me that this would be a good thing to put some debugging work into.

It might also be worthwhile to start a conversation on the babel-lodash plugin; have an extra set of eyes on the problem.

graingert commented 6 years ago

@bcoe pinged them in: https://github.com/lodash/babel-plugin-lodash/issues/207

pascalduez commented 6 years ago

Looks similar to: https://github.com/ant-design/babel-plugin-import/issues/189
https://github.com/ant-design/babel-plugin-import/issues/172

DoWhileGeek commented 6 years ago

i'm having incredibly similar issues with react-native-dotenv.

patsissons commented 6 years ago

Ran into this too with babel-plugin-dotenv-import (which is very similar to react-native-dotenv). The solution is to import all dotenv defines in a single file and then ignore that file for coverage:

    "coveragePathIgnorePatterns": [
      "<rootDir>/node_modules/",
      "<rootDir>/src/utils/env.js"
    ],