search

codeclimate / ruby-test-reporter

DEPRECATED Uploads Ruby test coverage data to Code Climate
https://codeclimate.com
Other
92 stars 92 forks source link

Coverage dffers locally and on CodeClimate #157

Closed SergeyKishenin closed 7 years ago

SergeyKishenin commented 7 years ago

Hi,

I'm getting different results locally and on CodeClimate for the same file:

screenshot from 2016-11-23 00 30 36

screenshot from 2016-11-23 01 11 44

And CodeClimate reports blank lines and end statements as uncovered, but locally it's skipped:

screenshot from 2016-11-23 01 12 08

We are using codeclimate-test-reporter v1.0.3 and Solano CI Service.

bodrovis commented 7 years ago

I was having a pretty similar issue, but after adding an after hook into the .travis.yml https://github.com/bodrovis/ok_linker/blob/master/.travis.yml#L11 results seem to be updated propertly

after_success:
  - bundle exec codeclimate-test-reporter
pbrisbin commented 7 years ago

@SergeyKishenin would you be able to grab the actual coverage/.resultset.json file that Simplecov produces right before you invoke the reporter on CI and compare it to what's produced when you do whatever you do to locally view results?

If they're not the same, then something different in your local vs CI environment is making Simplecov itself behave differently.

If they're identical though, then the problem is either in reporting or rendering.

Sometimes, seeing such a reduction in coverage on Code Climate can be caused by running multiple test suites. It's possible viewing locally will show merged coverage results from all suites, but our reporter is limited in that it will only report the coverage results from the last suite to run.

pbrisbin commented 7 years ago

I'm going to go ahead and close this as stale. Please re-open if you are still having this issue and are able to provide those additional details. Also note that #162 (now fixed) may also have been causing this.