search

istanbuljs / nyc

the Istanbul command line interface
https://istanbul.js.org/
ISC License
5.61k stars 360 forks source link

Constructors not covered 😦 #374

Closed erikras closed 8 years ago

erikras commented 8 years ago

Expected Behavior

The constructor of a class new'd many times in tests should be considered covered.

Observed Behavior

branch

Bonus Points! Code (or Repository) that Reproduces Issue

I first noticed it here: https://github.com/erikras/redux-form

Forensic Information

Operating System: Mac 10.11.6 Environment Information: https://gist.github.com/erikras/6cffc04ee8be7dcc5373572c42d90119

Additional Info

Issue suggested by @bcoe on Twitter.

Gudahtt commented 8 years ago

I've encountered the same issue using istanbul version 1.0.0-alpha.2 - I was about to try switching to nyc, but it looks like the same problem exists here too. It looks like babel-istanbul is having similar issues as well: https://github.com/jmcriffey/babel-istanbul/issues/72

This started occuring around the same time as the Babel 6.14.0 release, late last week.

Gudahtt commented 8 years ago

Sorry for the confusion; it turns out the errors that I experienced disappeared after switching to nyc. The similarity is still curious though - the problem was identical in appearance.

I was able to reproduce the error with redux-form. I was also able to solve it, by installing babel-plugin-istanbul and using that when generating the coverage report.

I had to adjust .babelrc to add a test environment where the istanbul plugin is used, then adjust the test:cov npm script in package.json to set NODE_ENV=test and set the nyc flags "instrument" and "sourceMap" to false - all as described in the nyc readme. After that, the uncovered branches dissapeared.

Edit: So I suppose, thinking back on this, that this points to a problem either in Babel 6.14.0's sourcemaps or in the istanbul instrumenter. The use of babel-plugin-istanbul precludes using the istanbul instrumenter, if I understand correctly. That would explain why the problem dissapeared once it was used.

bcoe commented 8 years ago

@erikras as @Gudahtt indicates, make sure you add:

  {"nyc": {
    "sourceMap": false,
    "instrument": false
  }}

to your package.json, I've tested with both native ES6 and babel, and in both cases the branch of the constructor was covered, my package.json looks like this:

{
  "name": "yarsay",
  "version": "1.0.5",
  "description": "Tell Pirate Joe what to say",
  "main": "lib/yarsay.js",
  "bin": "lib/cli.js",
  "scripts": {
    "build": "cd src; babel ./*.js -d ../lib",
    "coverage": "cat ./coverage/lcov.info | coveralls",
    "prepublish": "npm run build",
    "//pretest": "standard ./src/*.js",
    "test": "cross-env NODE_ENV=test nyc --silent mocha test/*.js",
    "posttest": "nyc report --reporter lcov --reporter text",
    "release": "standard-version"
  },
  "babel": {
    "presets": [
      "es2015"
    ],
    "env": {
      "test": {
        "plugins": [
          "istanbul"
        ]
      }
    }
  },
  "nyc": {
    "include": [
      "src/**/*.js"
    ],
    "exclude": [
      "src/foo/snuh.js"
    ],
    "require": [
      "babel-core/register"
    ],
    "sourceMap": false,
    "instrument": false
  },
  "repository": {
    "type": "git",
    "url": "http://github.com/yargs/yarsay.git"
  },
  "keywords": [
    "yargs",
    "cli",
    "yarsay",
    "pirate",
    "talk"
  ],
  "author": "Ben Coe <ben@npmjs.com>",
  "license": "ISC",
  "devDependencies": {
    "babel-cli": "^6.10.1",
    "babel-core": "^6.14.0",
    "babel-plugin-__coverage__": "^11.0.0",
    "babel-plugin-istanbul": "^2.0.0",
    "babel-preset-es2015": "^6.14.0",
    "chai": "^3.4.1",
    "coveralls": "^2.11.6",
    "cross-env": "^1.0.8",
    "i": "^0.3.5",
    "mocha": "^2.3.4",
    "npm": "^3.10.6",
    "nyc": "^8.1.0",
    "standard": "^7.1.2",
    "standard-version": "^2.3.0"
  },
  "dependencies": {
    "bluebird": "^3.1.1",
    "chalk": "^1.1.3",
    "cliui": "^3.1.0",
    "get-stdin": "^5.0.1",
    "window-size": "^0.2.0",
    "yargs": "^4.7.1"
  }
}
taylor1791 commented 8 years ago

I had the same problem and @bcoe's solution worked for me. I was using ava.

JaKXz commented 8 years ago

Just ensure that with nyc@8 your custom exclude array specifies node_modules etc.

erikras commented 8 years ago

Adding

  "nyc": {
    "include": [
      "src/**/*.js"
    ],
    "sourceMap": false,
    "instrument": false
  },

to my package.json results in:

----------|----------|----------|----------|----------|----------------|
File      |  % Stmts | % Branch |  % Funcs |  % Lines |Uncovered Lines |
----------|----------|----------|----------|----------|----------------|
All files |  Unknown |  Unknown |  Unknown |  Unknown |                |
----------|----------|----------|----------|----------|----------------|

Additional Info

To clarify, this is only happening locally on my mac. When the tests run in Travis, it's fine.

bcoe commented 8 years ago

@erikras is this the redux-form example again? there should be no difference between OSX and Ubuntu.

Gudahtt commented 8 years ago

@erikras Sorry I should have been more clear - the problem you're having can be solved by using the babel plugin babel-plugin-istanbul. redux-form does not appear to be using that plugin currently.

Setting sourceMap and instrument to false is required when you're using babel-plugin-istanbul, because babel-plugin-istanbul handles the instrumentation instead of nyc - so you need to disable nyc's instrumentation. Disabling nyc's instrumentation without adding babel-plugin-istanbul just results in... no instrumentation happening at all, so your coverage report is empty.

Check out the "Use with babel-plugin-istanbul for ES6/ES7/ES2015 Support" section of the nyc README for more information.

Gudahtt commented 8 years ago

@bcoe is nyc still intended to be used without babel-plugin-istanbul for projects that use babel?

The README states that babel-plugin-istanbul has better ES6/ES7/ES2015 support, but it seems to imply that it should still work without it, solely through the use of source maps. If that's the case, this bug is still valid. Though, from the look of things, the fault likely lies with Babel's source map generation, not this project. I can't say for sure though.

bcoe commented 8 years ago

@erikras I installed babel-plugin-istanbul:

npm i babel-plugin-istanbul --save-dev

updated .babelrc to look like this:

{
  "presets": ["es2015-no-commonjs", "react", "stage-2"],
  "env": {
    "development": {
      "plugins": ["lodash", "transform-es2015-modules-commonjs"]
    },
    "test": {
      "plugins": ["lodash", "transform-es2015-modules-commonjs", "istanbul"]
    },
    "production": {
      "plugins": ["lodash", "transform-es2015-modules-commonjs"]
    },
    "es": {
      "plugins": ["lodash", "./babel-lodash-es"]
    }
  }
}

and updated package.json to look like this:

{
  "name": "redux-form",
  "version": "6.0.2",
  "description": "A higher order component decorator for forms using Redux and React",
  "main": "./lib/index.js",
  "jsnext:main": "./es/index.js",
  "repository": {
    "type": "git",
    "url": "https://github.com/erikras/redux-form"
  },
  "scripts": {
    "build": "npm run build:lib && npm run build:es && npm run build:umd && npm run build:umd:min",
    "build:lib": "babel src --out-dir lib",
    "build:es": "cross-env BABEL_ENV=es babel src --out-dir es",
    "build:umd": "cross-env NODE_ENV=development webpack src/index.js dist/redux-form.js",
    "build:umd:min": "cross-env NODE_ENV=production webpack src/index.js dist/redux-form.min.js",
    "clean": "rimraf dist lib",
    "lint": "eslint src",
    "example:simple": "node examples/simple/server.js",
    "prepublish": "npm run test && npm run clean && npm run build",
    "test": "mocha --compilers js:babel-register --recursive --recursive \"src/**/__tests__/*\" --require src/__tests__/setup.js",
    "test:watch": "npm test -- --watch",
    "test:cov": "cross-env NODE_ENV=test nyc --reporter=lcov --reporter=text npm test",
    "test:codecov": "cat ./coverage/lcov.info | ./node_modules/codecov.io/bin/codecov.io.js"
  },
  "keywords": [
    "react",
    "reactjs",
    "flux",
    "redux",
    "react-redux",
    "redux-form",
    "form",
    "decorator"
  ],
  "author": "Erik Rasmussen <rasmussenerik@gmail.com> (http://github.com/erikras)",
  "license": "MIT",
  "bugs": {
    "url": "https://github.com/erikras/redux-form/issues"
  },
  "homepage": "https://redux-form.com/",
  "dependencies": {
    "array-findindex-polyfill": "^0.1.0",
    "deep-equal": "^1.0.1",
    "es6-error": "^3.0.0",
    "hoist-non-react-statics": "^1.0.5",
    "invariant": "^2.2.1",
    "is-promise": "^2.1.0",
    "lodash": "^4.12.0",
    "lodash-es": "^4.12.0",
    "shallowequal": "^0.2.2"
  },
  "devDependencies": {
    "babel-cli": "^6.3.17",
    "babel-core": "^6.3.26",
    "babel-eslint": "^6.0.4",
    "babel-loader": "^6.2.0",
    "babel-plugin-istanbul": "^2.0.0",
    "babel-plugin-lodash": "^3.1.2",
    "babel-plugin-syntax-async-functions": "^6.5.0",
    "babel-plugin-transform-es2015-modules-commonjs": "^6.8.0",
    "babel-plugin-transform-regenerator": "^6.6.5",
    "babel-preset-es2015-no-commonjs": "0.0.2",
    "babel-preset-react": "^6.1.18",
    "babel-preset-stage-2": "^6.1.18",
    "babel-register": "^6.3.13",
    "codecov.io": "^0.1.6",
    "cross-env": "^2.0.0",
    "eslint": "^3.3.1",
    "eslint-config-rackt": "^1.1.1",
    "eslint-plugin-react": "^6.1.2",
    "expect": "^1.14.0",
    "expect-element": "^1.1.1",
    "expect-immutable": "0.0.3",
    "expect-predicate": "^1.0.0",
    "flux-standard-action": "^0.6.1",
    "jsdom": "^9.1.0",
    "lodash-webpack-plugin": "^0.10.0",
    "mocha": "^3.0.2",
    "nyc": "^8.1.0",
    "react": "^15.3.1",
    "react-addons-test-utils": "^15.3.1",
    "react-dom": "^15.3.1",
    "react-redux": "^4.4.1",
    "redux": "^3.3.1",
    "redux-immutablejs": "0.0.8",
    "rifraf": "^2.0.2",
    "rimraf": "^2.5.1",
    "webpack": "^1.13.2"
  },
  "peerDependencies": {
    "react": "^15.0.0",
    "react-redux": "^4.3.0",
    "redux": "^3.0.0"
  },
  "files": [
    "README.md",
    "es",
    "lib",
    "dist",
    "immutable.js"
  ],
  "npmName": "redux-form",
  "npmFileMap": [
    {
      "basePath": "/dist/",
      "files": [
        "*.js"
      ]
    }
  ],
  "nyc": {
    "include": [
      "src/**/*.js"
    ],
    "sourceMap": false,
    "instrument": false
  }
}

Note the new test environment in both places.

This seemed to do the trick ... it looks like you have 100% line, branch, and function coverage, pretty awesome:

--------------------------------|----------|----------|----------|----------|----------------|
File                            |  % Stmts | % Branch |  % Funcs |  % Lines |Uncovered Lines |
--------------------------------|----------|----------|----------|----------|----------------|
All files                       |      100 |      100 |      100 |      100 |                |
 src                            |      100 |      100 |      100 |      100 |                |
  ConnectedField.js             |      100 |      100 |      100 |      100 |                |
  ConnectedFieldArray.js        |      100 |      100 |      100 |      100 |                |
  ConnectedFields.js            |      100 |      100 |      100 |      100 |                |
  Field.js                      |      100 |      100 |      100 |      100 |                |
  FieldArray.js                 |      100 |      100 |      100 |      100 |                |
  Fields.js                     |      100 |      100 |      100 |      100 |                |
  SubmissionError.js            |      100 |      100 |      100 |      100 |                |
  actionTypes.js                |      100 |      100 |      100 |      100 |                |
  actions.js                    |      100 |      100 |      100 |      100 |                |
  asyncValidation.js            |      100 |      100 |      100 |      100 |                |
  createAll.js                  |      100 |      100 |      100 |      100 |                |
  createFieldArrayProps.js      |      100 |      100 |      100 |      100 |                |
  createFieldProps.js           |      100 |      100 |      100 |      100 |                |
  defaultShouldAsyncValidate.js |      100 |      100 |      100 |      100 |                |
  deleteInWithCleanUp.js        |      100 |      100 |      100 |      100 |                |
  formValueSelector.js          |      100 |      100 |      100 |      100 |                |
  handleSubmit.js               |      100 |      100 |      100 |      100 |                |
  hasError.js                   |      100 |      100 |      100 |      100 |                |
  immutable.js                  |      100 |      100 |      100 |      100 |                |
  index.js                      |      100 |      100 |      100 |      100 |                |
  isReactNative.js              |      100 |      100 |      100 |      100 |                |
  propTypes.js                  |      100 |      100 |      100 |      100 |                |
  reducer.js                    |      100 |      100 |      100 |      100 |                |
  reduxForm.js                  |      100 |      100 |      100 |      100 |                |
  values.js                     |      100 |      100 |      100 |      100 |                |
 src/events                     |      100 |      100 |      100 |      100 |                |
  createOnBlur.js               |      100 |      100 |      100 |      100 |                |
  createOnChange.js             |      100 |      100 |      100 |      100 |                |
  createOnDragStart.js          |      100 |      100 |      100 |      100 |                |
  createOnDrop.js               |      100 |      100 |      100 |      100 |                |
  createOnFocus.js              |      100 |      100 |      100 |      100 |                |
  getValue.js                   |      100 |      100 |      100 |      100 |                |
  isEvent.js                    |      100 |      100 |      100 |      100 |                |
  silenceEvent.js               |      100 |      100 |      100 |      100 |                |
  silenceEvents.js              |      100 |      100 |      100 |      100 |                |
 src/selectors                  |      100 |      100 |      100 |      100 |                |
  getFormValues.js              |      100 |      100 |      100 |      100 |                |
  isDirty.js                    |      100 |      100 |      100 |      100 |                |
  isInvalid.js                  |      100 |      100 |      100 |      100 |                |
  isPristine.js                 |      100 |      100 |      100 |      100 |                |
  isValid.js                    |      100 |      100 |      100 |      100 |                |
 src/structure/immutable        |      100 |      100 |      100 |      100 |                |
  deepEqual.js                  |      100 |      100 |      100 |      100 |                |
  expectations.js               |      100 |      100 |      100 |      100 |                |
  index.js                      |      100 |      100 |      100 |      100 |                |
  splice.js                     |      100 |      100 |      100 |      100 |                |
 src/structure/plain            |      100 |      100 |      100 |      100 |                |
  deepEqual.js                  |      100 |      100 |      100 |      100 |                |
  deleteIn.js                   |      100 |      100 |      100 |      100 |                |
  expectations.js               |      100 |      100 |      100 |      100 |                |
  getIn.js                      |      100 |      100 |      100 |      100 |                |
  index.js                      |      100 |      100 |      100 |      100 |                |
  setIn.js                      |      100 |      100 |      100 |      100 |                |
  splice.js                     |      100 |      100 |      100 |      100 |                |
 src/util                       |      100 |      100 |      100 |      100 |                |
  getDisplayName.js             |      100 |      100 |      100 |      100 |                |
  shallowCompare.js             |      100 |      100 |      100 |      100 |                |
--------------------------------|----------|----------|----------|----------|----------------|
bcoe commented 8 years ago

@erikras feel free to reopen this ticket, if these fixes don't do the trick.

erikras commented 8 years ago

Finally got around to looking at this. Your fix is working. 👍 ❤️