pjcj / Devel--Cover

Code coverage metrics for Perl
http://www.pjcj.net/perl.html
93 stars 89 forks source link

Coverage is merged to one file if there are two files with exactly the same content #322

Open perlpunk opened 1 year ago

perlpunk commented 1 year ago

When there are to files with the following content:

# foo.pm
my $var = 'test';
# bar/baz.pm
my $var = 'test';

and my test looks like this:

# t/cover.t
use Test::More tests => 2;
use FindBin '$Bin';

ok require "$Bin/../lib/foo.pm", 'foo';
ok require "$Bin/../lib/bar/baz.pm", 'bar/baz';

then only one of them gets coverage:

% PERL5OPT=-MDevel::Cover prove t/cover.t
% cover -report html_minimal  cover_db
-------------- ------ ------ ------ ------ ------ ------ ------
File             stmt   bran   cond    sub    pod   time  total
-------------- ------ ------ ------ ------ ------ ------ ------
/usr/bin/prove  100.0   50.0    n/a  100.0    n/a   42.5   87.5
lib/foo.pm      100.0    n/a    n/a  100.0    n/a    0.0  100.0
t/cover.t       100.0    n/a    n/a  100.0    n/a   57.3  100.0
Total           100.0   50.0    n/a  100.0    n/a  100.0   94.8
-------------- ------ ------ ------ ------ ------ ------ ------

Is this a limitation of Devel::Cover or a bug?

Background: Of course it's not a very common thing to have two perl files with the same content under different locations. In our case we were able to simply make the modules distinct. So we have two perl files that are not part of the app we are testing, but rather our app can load modules, and we want to make sure that our test modules under the t/data/tests directory are covered as well.

It took us a while to find out why only ever one of the two modules would appear in the coverage report, so I thought I'll create this issue to see if it would be possible to fix it.

Reproducer repo: https://github.com/perlpunk/devel-cover-bug Run make cover

pjcj commented 1 year ago

Is this a limitation of Devel::Cover or a bug?

It's actually a feature :)

But perhaps it's a slightly too aggressive feature. The reason for doing this is because traditionally authors would put their modules in the lib directory. Then, as part of the build process the modules would get copied to blib. But for many modules the tests would sometimes use the modules in lib and sometimes in blib. So Devel::Cover took the stance that if you have identical files it doesn't matter which you use and the coverage of the files is merged. There's a prefer_lib option to report against lib in this case. Otherwise the order is sort of random, as you have discovered.

This behaviour is somewhat built-in. It could be changed, of course, perhaps to only consider files with the same name, or to be turned off, but that would take a little effort and, since you have a simple workaround, perhaps that would be a low priority.