The test suite output sometimes contain warnings which do not fail the tests, while they may indicate trouble. This may cause basic checking of the test results on GitHub Actions or CPAN Testers indicate all's well, while instead we could notice and fix bugs or corner cases early.
For example currently there are numerous warnings about the following when running on Windows with <=perl-5.22:
Unsuccessful stat on filename containing newline
Unsuccessful lstat on filename containing newline
Describe the solution you would like
I'd like the test suite to fail upon warnings in as much cases as possible. We already use Test::Warnings in some places, perhaps we can add it to as much test files as possible.
Describe alternatives you have considered
I checked Test::NoWarnings, though Test::Warnings seems to supersede it with functionality, and we already have that as a test dependency anyway.
It may be enough to add Test::Warnings to tests which may known to be throwing warnings currently, though that would defeat the purpose to detect unexpected warnings later elsewhere.
As
maintainer
I would like to
fail the test suite on warnings
so I can
notice and fix problems earlier
Additional context
The test suite output sometimes contain warnings which do not fail the tests, while they may indicate trouble. This may cause basic checking of the test results on GitHub Actions or CPAN Testers indicate all's well, while instead we could notice and fix bugs or corner cases early.
For example currently there are numerous warnings about the following when running on Windows with <=perl-5.22:
Describe the solution you would like
I'd like the test suite to fail upon warnings in as much cases as possible. We already use Test::Warnings in some places, perhaps we can add it to as much test files as possible.
Describe alternatives you have considered
I checked Test::NoWarnings, though Test::Warnings seems to supersede it with functionality, and we already have that as a test dependency anyway.
It may be enough to add Test::Warnings to tests which may known to be throwing warnings currently, though that would defeat the purpose to detect unexpected warnings later elsewhere.