It's possible to configure some tests to be skipped when smoking.
Depending on which test is to be skipped it uses a different implementation:
for tests in t/ it alters the test file
for tests in lib/, ext/, cpan/, dist/ it doesn't change the test file but instead it modifies the MANIFEST file that is in the root of the build directory.
This second method is less optimal, on my smoker I skipped a test in dist/ (because it fails on the system for unrelated reason);
The resulting smoke report:
Tests skipped on user request:
dist/Time-HiRes/t/utime.t
Failures: (common-args) none
[default]
../t/porting/manifest.t.....................................FAILED
10388, 10390-10391
What happened:
Test::Smoke::Smoker::set_skip_tests modified the MANIFEST file
t/porting/manifest.t checks if the MANIFEST contains all files; that is no longer the case since it was modified to remove some files from it.
I can think of several solutions (in no particular order)
Do nothing and leave it up to the user to also skip t/porting/manifest.t when skipping tests
Automatically skip t/porting/manifest.t when a test is being skipped
Alter Test::Smoke::Smoker::set_skip_tests to always modify the test file (i.e. remove the special case for lib|ext|cpan|dist)
t/harness which is used when running make test_harness does have some options to skip tests; but that might not be exposed via make test_harness (and isn't a solution for make test)
It's possible to configure some tests to be skipped when smoking.
Depending on which test is to be skipped it uses a different implementation:
t/it alters the test filelib/,ext/,cpan/,dist/it doesn't change the test file but instead it modifies the MANIFEST file that is in the root of the build directory.This second method is less optimal, on my smoker I skipped a test in dist/ (because it fails on the system for unrelated reason); The resulting smoke report:
What happened:
Test::Smoke::Smoker::set_skip_testsmodified the MANIFEST fileI can think of several solutions (in no particular order)
Test::Smoke::Smoker::set_skip_teststo always modify the test file (i.e. remove the special case forlib|ext|cpan|dist)t/harnesswhich is used when runningmake test_harnessdoes have some options to skip tests; but that might not be exposed viamake test_harness(and isn't a solution formake test)