Closed linearregression closed 8 years ago
ping???
I'm a bit confused by the usage of relative paths here. How are you planning to run these? My suspicion, aside from cover being broken, is that the paths are wrong (rebar3
never cd
s anywhere and stays at the root dir) and therefore cannot be either excluded nor included.
it used to work for old version of rebar and oldver version of erlang 17 (with rebar 3 older version a few months back last year, around Oct 2015 timeframe) Those relative path entries are from coverspec https://github.com/priestjim/gen_rpc/blob/master/test/gen_rpc.coverspec rebar.config is referencing the coverspec file as in https://github.com/priestjim/gen_rpc/blob/master/rebar.config
rebar3 is a whole new tool. Those relative paths are not going to work with rebar3 because it does not change directories while running like rebar did.
I thought rebar3 just read the coverspec in rebar.config and give it to underlying coverage functionality of common test??
Yes, but that doesn't change the fact that ../
is different in rebar3 than rebar2 because it does not change directories anymore.
So it means I should try putting the coverspec as root directory of project and see if it excludes? Actually I have another project but internal that does just that (not using relative path) and code coverage execution still ignores all the exclusion criteria.
when using rebar3 and ct the relevant beam files that ct works with are in '_build/PROFILE/lib/*/ebin'. try excluding those directories
Yes tried that and also using absolute path. No luck. Still including the test beams
ok. i will take a closer look at this over the weekend
When setting {cover_enabled,true}
in rebar.config, it seems rebar3 will perform its own cover compilation/analysis, independent of common_test. The coverspec file given to common_test is not read by rebar3 at all, so files included/excluded there are not honoured by rebar3's cover analysis. I don't know if it is possible to specify which modules rebar3 should cover compile and not, but by default it seems that it takes all beam files found in ebin and test.
If you want common_test to do the cover analysis, then set cover_enabled
to false
(default) in rebar.config, and only keep the cover
option to common_test, pointing out the coverspec file.
The cover result can then be found in the HTML logs from common_test.
@tsloughter thanks for updating the docs to explain rebar3's behavior when it comes to coverage. However, the question remains: why does rebar3 account for test code coverage INSIDE the test files? Can we do something about it?
@priestjim are you trying to exclude coverage statistics from the rebar3 cover tool, or from the ct cover tool? they are two different tools.
rebar3 just passes through the ct cover spec to ct. you'll have to ensure that the directories you are excluding are the ones in the _build
directory, as that is where rebar3 loads all modules from
if you are asking how to exclude coverage stats from the build in cover tool, you currently can't. it could be added as an option in the future. is that what you are asking for?
I am trying to exclude coverage accounting and reporting for the test files (ct and eunit)
i've made a pr that fixes the paths in your cover spec. let me know if this fixes your issue
can you explain a little bit what you mean by proper coverage? i have zero plans to support ct coverspecs in the rebar3 cover tool (but note that it works fine if you use the ct cover tool instead) but i would like to add a way to only include modules that are part of the applications being tested (so modules that are listed in the mod key of all .app files). would that fix your problem?
Yes it would. Essentially, when I use rebar3 to generate code coverage information, it takes into account ALL the modules available to the system, including the ones in the test/ directory, which is kind of pointless. Evaluating only the modules listed in the .app files would definitely fix the problem.
ok. i probably won't get to it until later this week, but i will see what i can do
I was able to unwillingly generate coverage data for _SUITE.erl
by using {src_dirs, ["test", "test/util"]}
instead of {extra_src_dirs, ["test/util"]}
. Removing _build
and changing to {extra_src_dirs, _}
got rid of that problem.
This is (somewhat?) related to #1179.
If you declare test files to be source files (by putting them in src_dirs) then we will generate coverage report for them since they're seen as source files. This would be a case of "this is not a bug, it's a feature."
If you declare test files to be source files (by putting them in src_dirs) then we will generate coverage report for them since they're seen as source files. This would be a case of "this is not a bug, it's a feature."
Just wrote it here in case somebody gets bit like me 😄
We are using rebar3, and run with coder coverage. It will add tests as part of coverage skeing coverage result!.
Our rebar.config https://github.com/priestjim/gen_rpc/blob/master/rebar.config this point to cover spec at: https://github.com/priestjim/gen_rpc/blob/master/test/gen_rpc.coverspec
Originally reported here https://github.com/priestjim/gen_rpc/issues/42