pjcj / Devel--Cover

Code coverage metrics for Perl
http://www.pjcj.net/perl.html
93 stars 89 forks source link

Silent mode doesn't suppress "Deleting old coverage for changed file" message #329

Open martinvonwittich opened 1 year ago

martinvonwittich commented 1 year ago

Devel::Cover has a silent option that is enabled by default when enabling Devel::Cover via PERL5OPT:

https://metacpan.org/pod/Devel::Cover

The -silent option is turned on when Devel::Cover is invoked via $HARNESS_PERL_SWITCHES or $PERL5OPT.

Contrary to my expectations, this doesn't suppress the "Deleting old coverage for changed file" message whenever I change my code without manually deleting the old coverage data, which causes spurious test breakage because my tests check the STDERR output of my program:

martin.mein-iserv.de chkmanagedusers (63013-chkmanagedusers) # ./chkmanagedusers.bats -f maintained
chkmanagedusers.bats
 ✗ invalid: user with explicitly maintained temp password (dry-run)
   tags: invalid
   (from function `assert_equal' in file test_helper/bats-assert/src/assert_equal.bash, line 40,
    from function `test_chkmanagedusers' in file chkmanagedusers.bats, line 105,
    in test file chkmanagedusers.bats, line 475)
     `TEST_CONFIG=invalid-user-with-maintained-temp-password TEST_REPAIR=0 TEST_EXITCODE=0 test_chkmanagedusers' failed

   -- values do not equal --
   expected (1 lines):
     user chkmanagedusers-bats-uwiotp: password/temp: true requires maintain_password: initial_only
   actual (2 lines):
     user chkmanagedusers-bats-uwiotp: password/temp: true requires maintain_password: initial_only
     Devel::Cover: Deleting old coverage for changed file /usr/sbin/chkmanagedusers
   --

 ✓ invalid: user with explicitly maintained temp password (repair)
 ✓ invalid: user with implicitly maintained temp password (dry-run)
 ✓ invalid: user with implicitly maintained temp password (repair)

4 tests, 1 failure

I've tracked this warning down to https://github.com/pjcj/Devel--Cover/blob/82526a3d8b9886ec579896311e66f4556a3b41cd/lib/Devel/Cover/DB/Structure.pm#L306-L307

This warning should probably honor $Devel::Cover::Silent.

pjcj commented 6 months ago

Thanks for the report. I suspect you're right. I wonder whether I thought the warning was sufficiently important to show it anyway. And I really don't want to go down the -really-silent path...

jsoref commented 2 months ago

I've definitely tripped on this... I kinda guessed that was the reason (but didn't look for this ticket) at the time...

If it's relatively safe, I'd probably favor a "log" file (in the same directory as the coverage database) that lists each time coverage is deleted for a file... That'd leave the information available, but not surface it (thus avoiding breaking tests, while enabling people to debug broken programs/tests).