Pod::Html: t/htmldir3.t test produces spurious pass

[p5pRT]

Migrated from rt.perl.org#114126 (status was 'open')

Searchable as RT114126$

[p5pRT]

From @jkeenan

This is a bug report for perl from jkeenan@​cpan.org\, generated with the help of perlbug 1.39 running under perl 5.16.0.

In the course of working on my fork of Pod-Html\, I have come to a point where all the code has been refactored into an object-oriented interface and where all the original tests (revised to have the OO interface) are passing -- except one.

Further investigation suggests that the one failing test is *only accidentally passing* in the test suite found in ext/Pod-Html/t/ in the Perl 5 core distribution.

You can see this for yourself by cd-ing to ext/Pod-Html/ in blead. Call​:

##### prove -Ilib t/*.t #####

All tests should pass.

Now\, copy into ext/Pod-Html/t/ the two files attached to this RT​: xhtmldir3.t and yhtmldir3.t. xhtmldir3.t simply removes the first of two invocations of the testing function convert_n_test() from xhtmldir3.t. yhtmldir3.t switches the order of those two invocations of convert_n_test().

If all the tests and testing functions were well formulated in the first place\, you would expect all tests to pass. Now call​:

##### prove -Ilib t/*.t #####

The sole test in xhtmldir3.t and the first test in yhtmldir3.t should now fail\, with output like this​:

##### t/xhtmldir3.t .. 1/? # Failed test 'test --htmldir and --htmlroot 3b' # at t/pod2html-lib.pl line 78. # --- htmldir3_expected.tmp 2012-07-12 13​:15​:14.000000000 -0400 # +++ /Users/jimk/gitwork/perl/ext/Pod-Html/t/htmldir3.html 2012-07-12 13​:15​:14.000000000 -0400 # @​@​ -24\,7 +24\,7 @​@​ # # \

Normal text\, a \link\ to nowhere\,\

# # -\

a link to \<a href="Users/jimk/gitwork/perl/ext/Pod-Html/testdir/test.lib/var-copy.html">var-copy\\,\

# +\

a link to \var-copy\\,\

# # \

\<a href="Users/jimk/gitwork/perl/ext/Pod-Html/t/htmlescp.html">htmlescp\\,\

# # Looks like you failed 1 test of 1. t/xhtmldir3.t .. Dubious\, test returned 1 (wstat 256\, 0x100) Failed 1/1 subtests t/yhtmldir3.t .. 1/2 # Failed test 'test --htmldir and --htmlroot 3b' # at t/pod2html-lib.pl line 78. # --- htmldir3_expected.tmp 2012-07-12 13​:15​:14.000000000 -0400 # +++ /Users/jimk/gitwork/perl/ext/Pod-Html/t/htmldir3.html 2012-07-12 13​:15​:14.000000000 -0400 # @​@​ -24\,7 +24\,7 @​@​ # # \

Normal text\, a \link\ to nowhere\,\

# # -\

a link to \<a href="Users/jimk/gitwork/perl/ext/Pod-Html/testdir/test.lib/var-copy.html">var-copy\\,\

# +\

a link to \var-copy\\,\

# # \

\<a href="Users/jimk/gitwork/perl/ext/Pod-Html/t/htmlescp.html">htmlescp\\,\

# t/yhtmldir3.t .. 2/2 # Looks like you failed 1 test of 2. t/yhtmldir3.t .. Dubious\, test returned 1 (wstat 256\, 0x100) Failed 1/2 subtests

Test Summary Report

---

t/xhtmldir3.t (Wstat​: 256 Tests​: 1 Failed​: 1)   Failed test​: 1   Non-zero exit status​: 1 t/yhtmldir3.t (Wstat​: 256 Tests​: 2 Failed​: 1)   Failed test​: 1   Non-zero exit status​: 1 Files=19\, Tests=37\, 15 wallclock secs ( 0.36 usr 0.21 sys + 8.94 cusr   2.54 csys = 12.05 CPU) Result​: FAIL #####

The differences between the expected HTML and the generated HTML which are being reported are exactly the same as in the one failing test in my github branch.

##### # -\

a link to \<a href="Users/jimk/gitwork/perl/ext/Pod-Html/testdir/test.lib/var-copy.html">var-copy\\,\

# +\

a link to \var-copy\\,\

#####

This suggests to me that the problem is in the way t/htmldir3.t is formulated. My hunch is that the first test in t/htmldir3.t changes the environment in some way\, not yet understood\, that enables the second test to pass. But when we make that test the first one in a file\, it fails.

Thank you very much. Jim Keenan

---

Flags​:   category=library   severity=low   module=Pod​::Html

---

Site configuration information for perl 5.16.0​:

Configured by jimk at Sun May 20 20​:01​:26 EDT 2012.

Summary of my perl5 (revision 5 version 16 subversion 0) configuration​:

  Platform​:   osname=darwin\, osvers=8.11.0\, archname=darwin-2level   uname='darwin macintosh-8.local 8.11.0 darwin kernel version 8.11.0​: wed oct 10 18​:26​:00 pdt 2007; root​:xnu-792.24.17~1release_ppc power macintosh powerpc '   config_args='-des'   hint=recommended\, useposix=true\, d_sigaction=define   useithreads=undef\, usemultiplicity=undef   useperlio=define\, d_sfio=undef\, uselargefiles=define\, usesocks=undef   use64bitint=undef\, use64bitall=undef\, uselongdouble=undef   usemymalloc=n\, bincompat5005=undef   Compiler​:   cc='cc'\, ccflags ='-fno-common -DPERL_DARWIN -fno-strict-aliasing -pipe -I/usr/local/include -I/opt/local/include'\,   optimize='-O3'\,   cppflags='-fno-common -DPERL_DARWIN -fno-strict-aliasing -pipe -I/usr/local/include -I/opt/local/include'   ccversion=''\, gccversion='4.0.1 (Apple Computer\, Inc. build 5250)'\, gccosandvers=''   intsize=4\, longsize=4\, ptrsize=4\, doublesize=8\, byteorder=4321   d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=16   ivtype='long'\, ivsize=4\, nvtype='double'\, nvsize=8\, Off_t='off_t'\, lseeksize=8   alignbytes=8\, prototype=define   Linker and Libraries​:   ld='env MACOSX_DEPLOYMENT_TARGET=10.3 cc'\, ldflags =' -L/usr/local/lib -L/opt/local/lib'   libpth=/usr/local/lib /opt/local/lib /usr/lib   libs=-ldbm -ldl -lm -lc   perllibs=-ldl -lm -lc   libc=\, so=dylib\, useshrplib=false\, libperl=libperl.a   gnulibc_version=''   Dynamic Linking​:   dlsrc=dl_dlopen.xs\, dlext=bundle\, d_dlsymun=undef\, ccdlflags=' '   cccdlflags=' '\, lddlflags=' -bundle -undefined dynamic_lookup -L/usr/local/lib -L/opt/local/lib'

Locally applied patches​:

---

@​INC for perl 5.16.0​:   /usr/local/lib/perl5/site_perl/5.16.0/darwin-2level   /usr/local/lib/perl5/site_perl/5.16.0   /usr/local/lib/perl5/5.16.0/darwin-2level   /usr/local/lib/perl5/5.16.0   /usr/local/lib/perl5/site_perl/5.14.2   /usr/local/lib/perl5/site_perl/5.14.0   /usr/local/lib/perl5/site_perl/5.12.0   /usr/local/lib/perl5/site_perl/5.10.1   /usr/local/lib/perl5/site_perl/5.10.0   /usr/local/lib/perl5/site_perl   .

---

Environment for perl 5.16.0​:

DYLD_LIBRARY_PATH=/Users/jimk/work/pseudoinstall/lib​:/Users/jimk/gitwork/parrot/blib/lib   HOME=/Users/jimk   LANG (unset)   LANGUAGE (unset)   LD_LIBRARY_PATH (unset)   LOGDIR (unset)

PATH=/usr/local/bin​:/opt/local/bin​:/opt/local/sbin​:/usr/local/bin​:/opt/local/bin​:/opt/local/sbin​:/bin​:/sbin​:/usr/bin​:/usr/sbin​:/Users/jimk/bin​:/Users/jimk/bin/perl​:/Users/jimk/bin/c​:/Users/jimk/bin/shell​:/sw/lib​:/sw/bin​:/Users/jimk/bin​:/Users/jimk/bin/perl​:/Users/jimk/bin/c​:/Users/jimk/bin/shell​:/sw/lib​:/sw/bin   PERL_BADLANG (unset)   SHELL=/bin/bash

[p5pRT]

From @jkeenan

#!/usr/bin/perl -w # -*- perl -*-

BEGIN {   require "t/pod2html-lib.pl"; }

END {   rem_test_dir(); }

use strict; use Cwd; use File​::Spec; use File​::Spec​::Functions; use Test​::More tests => 2;

SKIP​: {   my $output = make_test_dir();   skip "$output"\, 2 if $output;

  my $cwd = cwd();   my ($v\, $d) = splitpath($cwd\, 1);   my $relcwd = substr($d\, length(File​::Spec->rootdir()));

  my $data_pos = tell DATA; # to read \ twice

  convert_n_test("htmldir3"\, "test --htmldir and --htmlroot 3b"\,   "--podpath=". catdir($relcwd\, 't')\,   "--podroot=$v". File​::Spec->rootdir\,   "--htmldir=t"\,   "--outfile=t/htmldir3.html"\,   "--quiet"\,   );

  seek DATA\, $data_pos\, 0; # to read \ twice (expected output is the same)

  convert_n_test("htmldir3"\, "test --htmldir and --htmlroot 3a"\,   "--podpath=$relcwd"\,   "--podroot=$v". File​::Spec->rootdir\,   "--htmldir=". catdir($cwd\, 't'\, '')\, # test removal trailing slash\,   "--quiet"\,   ); }

__DATA__ \<?xml version="1.0" ?> \<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http​://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> \ \ \ \<meta http-equiv="content-type" content="text/html; charset=utf-8" /> \ \

\

\

  \
• \NAME\\
  \
• \LINKS\\
\

\

NAME\

\

htmldir - Test --htmldir feature\

\

LINKS\

\

Normal text\, a \link\ to nowhere\,\

\

a link to \var-copy\\,\

\

\htmlescp\\,\

\

\"Another Head 1" in feature\\,\

\

and another \"Another Head 1" in feature\.\

\

\

[p5pRT]

From @jkeenan

#!/usr/bin/perl -w # -*- perl -*-

BEGIN {   require "t/pod2html-lib.pl"; }

END {   rem_test_dir(); }

use strict; use Cwd; use File​::Spec; use File​::Spec​::Functions; use Test​::More qw(no_plan); # tests => 2;

SKIP​: {   my $output = make_test_dir();   skip "$output"\, 1 if $output;

  my $cwd = cwd();   my ($v\, $d) = splitpath($cwd\, 1);   my $relcwd = substr($d\, length(File​::Spec->rootdir()));

  convert_n_test("htmldir3"\, "test --htmldir and --htmlroot 3b"\,   "--podpath=". catdir($relcwd\, 't')\,   "--podroot=$v". File​::Spec->rootdir\,   "--htmldir=t"\,   "--outfile=t/htmldir3.html"\,   "--quiet"\,   ); }

__DATA__ \<?xml version="1.0" ?> \<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http​://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> \ \ \ \<meta http-equiv="content-type" content="text/html; charset=utf-8" /> \ \

\

\

  \
• \NAME\\
  \
• \LINKS\\
\

\

NAME\

\

htmldir - Test --htmldir feature\

\

LINKS\

\

Normal text\, a \link\ to nowhere\,\

\

a link to \var-copy\\,\

\

\htmlescp\\,\

\

\"Another Head 1" in feature\\,\

\

and another \"Another Head 1" in feature\.\

\

\

[p5pRT]

From @nwc10

On Fri\, Jul 13\, 2012 at 05​:35​:19AM -0700\, James E Keenan wrote​:

Further investigation suggests that the one failing test is *only accidentally passing* in the test suite found in ext/Pod-Html/t/ in the Perl 5 core distribution.

This suggests to me that the problem is in the way t/htmldir3.t is formulated. My hunch is that the first test in t/htmldir3.t changes the environment in some way\, not yet understood\, that enables the second test to pass. But when we make that test the first one in a file\, it fails.

I'm not clear what you'd like to happen as a result of this ticket.

Is it a request for someone else to investigate to give a second opinion on the cause\, and to to propose a solution?

Or is this more a "note to self" to come back and fix it at some point?

Nicholas Clark

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @jkeenan

On Mon Jul 23 02​:18​:36 2012\, nicholas wrote​:

On Fri\, Jul 13\, 2012 at 05​:35​:19AM -0700\, James E Keenan wrote​:

Further investigation suggests that the one failing test is *only accidentally passing* in the test suite found in ext/Pod-Html/t/ in the Perl 5 core distribution.

This suggests to me that the problem is in the way t/htmldir3.t is formulated. My hunch is that the first test in t/htmldir3.t changes the environment in some way\, not yet understood\, that enables the second test to pass. But when we make that test the first one in a file\, it fails.

I'm not clear what you'd like to happen as a result of this ticket.

Is it a request for someone else to investigate to give a second opinion on the cause\, and to to propose a solution?

Or is this more a "note to self" to come back and fix it at some point?

Both.

At the hackathon held at DuckDuckGo in Pennsylvania last month\, I had occasion to discuss Pod​::Html with Ricardo Signes. He suggested that it would worth considering "dual-lifing" this module and I set up a fork on my github to explore what this would look like. In the course of that exploration I've encountered several issues for which I've opened RTs. While I've proposed patches for some of them\, there's a lot about Pod​::Html I don't yet understand. For those cases I'm flagging the issues in the hope that someone else can diagnose the problem sooner than I can.

In this particular case\, it would be good to (a) get an explanation of what the purpose of the misleading test is; (b) develop an understanding of what is changing in the testing environment that produces the spurious pass.

Thank you very much. Jim Keenan

[p5pRT]

From @rjbs

* James E Keenan via RT \perlbug\-followup@&#8203;perl\.org [2012-07-23T08​:03​:47]

In this particular case\, it would be good to (a) get an explanation of what the purpose of the misleading test is; (b) develop an understanding of what is changing in the testing environment that produces the spurious pass.

It would be good to get the input of Marc Green\, who has probably done the most fiddling with Pod​::Html recently.

cc'd.

via perlbug​: queue​: perl5 status​: open https://rt-archive.perl.org/perl5/Ticket/Display.html?id=114126

-- rjbs

[p5pRT]

From @jkeenan

On Mon Jul 23 05​:03​:47 2012\, jkeenan wrote​:

On Mon Jul 23 02​:18​:36 2012\, nicholas wrote​:

On Fri\, Jul 13\, 2012 at 05​:35​:19AM -0700\, James E Keenan wrote​:

Further investigation suggests that the one failing test is *only accidentally passing* in the test suite found in ext/Pod-Html/t/ in the Perl 5 core distribution.

This suggests to me that the problem is in the way t/htmldir3.t is formulated. My hunch is that the first test in t/htmldir3.t changes the environment in some way\, not yet understood\, that enables the second test to pass. But when we make that test the first one in a file\, it fails.

I'm not clear what you'd like to happen as a result of this ticket.

Is it a request for someone else to investigate to give a second opinion on the cause\, and to to propose a solution?

Or is this more a "note to self" to come back and fix it at some point?

Both.

At the hackathon held at DuckDuckGo in Pennsylvania last month\, I had occasion to discuss Pod​::Html with Ricardo Signes. He suggested that it would worth considering "dual-lifing" this module and I set up a fork on my github to explore what this would look like. In the course of that exploration I've encountered several issues for which I've opened RTs. While I've proposed patches for some of them\, there's a lot about Pod​::Html I don't yet understand. For those cases I'm flagging the issues in the hope that someone else can diagnose the problem sooner than I can.

In this particular case\, it would be good to (a) get an explanation of what the purpose of the misleading test is; (b) develop an understanding of what is changing in the testing environment that produces the spurious pass.

Thank you very much. Jim Keenan

Two calendar years later ... and this just got weirder!

I created a branch on the dromedary test server today and added the two test files (with minor updatings) originally posted in this ticket.

I then configure\, built and tested perl in that branch. As expected\, the two additional files failed.

I then repeatedly ran this set of commands​:

##### cd t;./perl harness -v ../ext/Pod-Html/t/*.t; cd- #####

... and got different results each time!

t/xhtmldir3.t and t/yhtmldir3.t failed each time\, as expected. But each repetition generated failures in 0 to 2 *other*\, previously passing files! The output is in the attachment.

Thank you very much. Jim Keenan

[p5pRT]

From @jkeenan

On dromedary​: branch​: 114126-pod-html-accidental-pass Forked from blead at 2d66f61e3a78c413bbb02ffa1bab2ad8aa6aed5a

Test Summary Report

---

../ext/Pod-Html/t/xhtmldir3.t (Wstat​: 256 Tests​: 1 Failed​: 1)   Failed test​: 1   Non-zero exit status​: 1 ../ext/Pod-Html/t/yhtmldir3.t (Wstat​: 256 Tests​: 2 Failed​: 1)   Failed test​: 1   Non-zero exit status​: 1 Files=2366\, Tests=683642\, 124 wallclock secs (71.71 usr 7.55 sys + 441.97 cusr 38.76 csys = 559.99 CPU) Result​: FAIL make​: *** [test_harness] Error 2

okay\, that's per #114126.

But now let's run the harness over that directory​:

$ cd t;./perl harness -v ../ext/Pod-Html/t/*.t;cd - ===( 1;0 1/1 0/? 0/? 0/? 0/? 0/3 0/? 0/? )================ok ===( 1;0 0/? 0/? 0/? 0/? 0/3 0/? 0/? )=====================Directory 'test.lib' exists (it shouldn't yet) - removing it at t/pod2html-lib.pl line 14. ===( 2;0 1/10 0/? 0/1 0/1 0/3 0/? 0/1 0/? )==============Directory 'test.lib' exists (it shouldn't yet) - removing it at t/pod2html-lib.pl line 14. ===( 4;0 2/10 0/1 1/1 0/1 0/3 0/? 0/1 0/? )============== # Failed test 'Cache created' # at t/cache.t line 33. # got​: undef # expected​: '1' Cannot open cache file​: No such file or directory at t/cache.t line 34. # Looks like you planned 10 tests but ran 3. # Looks like you failed 1 test of 3 run. # Looks like your test exited with 2 just after 3. ===( 6;0 3/10 0/1 1/1 1/1 0/3 0/? 0/1 0/? )==============ok ===( 6;0 3/10 0/1 1/1 0/3 0/? 0/1 0/? )===================Dubious\, test returned 2 (wstat 512\, 0x200) Failed 8/10 subtests ===( 6;0 0/1 1/1 0/3 0/? 0/1 0/? 0/? )=====================ok ===( 8;0 0/1 1/3 0/2 1/1 0/? 0/? 0/? 0/? )================ok ===( 9;0 0/1 2/3 0/2 0/? 0/? 0/? 0/? )=====================ok ===( 10;0 0/1 0/2 0/? 0/? 0/? 0/? 0/? )===================== # Failed test 'cross references' # at t/pod2html-lib.pl line 82. # --- crossref_expected.tmp 2014-01-01 21​:23​:59.768167006 +0100 # +++ /home/jkeenan/perl/ext/Pod-Html/t/crossref.html 2014-01-01 21​:23​:59.767167007 +0100 # @​@​ -35\,15 +35\,15 @​@​ #
# \

\"non existant section"\\

#
# -\

\var-copy\\

# +\

\var-copy\\

#
# -\

\"$"" in var-copy\\

# +\

\"$"" in var-copy\\

#
# \

\var-copy\\

#
# \

\var-copy/$"\\

#
# -\

\"First​:" in podspec-copy\\

# +\

\"First​:" in podspec-copy\\

#
# \

\podspec-copy/First​:\\

#
# Looks like you failed 1 test of 1. ===( 11;0 1/1 0/2 0/? 0/? 0/? 0/? 0/? 0/? )================Dubious\, test returned 1 (wstat 256\, 0x100) Failed 1/1 subtests ===( 12;0 1/2 0/? 0/? 0/? 0/? 0/? 0/? 0/? )================ok ===( 14;0 1/2 0/? 0/? 0/? 0/? 0/? 0/? 0/? )================ok ===( 15;0 0/? 0/? 0/? 0/? 0/? 0/? 0/? )=====================Directory 'test.lib' exists (it shouldn't yet) - removing it at t/pod2html-lib.pl line 14. ===( 20;0 2/3 1/2 1/2 0/1 1/1 0/1 0/1 0/? )================ok ===( 22;0 3/3 1/2 2/2 0/1 0/1 0/1 0/? )=====================ok ===( 22;0 1/2 2/2 0/1 0/1 0/1 0/? 0/? )=====================ok ===( 24;0 2/2 1/1 0/1 0/1 0/? 0/? 0/? 0/? )================ok ===( 24;0 1/1 0/1 0/1 0/? 0/? 0/? 0/? )=====================ok ===( 26;0 1/1 1/1 0/? 0/? 0/? 0/? )==========================ok ===( 26;0 1/1 0/? 0/? 0/? 0/? )===============================ok ===( 27;0 1/1 0/? 0/? 0/? )====================================ok ===( 27;1 0/? 0/? 0/? )=========================================Directory 'test.lib' exists (it shouldn't yet) - removing it at t/pod2html-lib.pl line 14. ===( 28;1 1/1 0/? 0/2 )=========================================ok ===( 28;1 0/? 0/2 )============================================== # Failed test 'test --htmldir and --htmlroot 3b' # at t/pod2html-lib.pl line 82. ===( 29;1 1/? 0/2 )==============================================# --- htmldir3_expected.tmp 2014-01-01 21​:24​:00.043166881 +0100 # +++ /home/jkeenan/perl/ext/Pod-Html/t/htmldir3.html 2014-01-01 21​:24​:00.043166880 +0100 # @​@​ -24\,7 +24\,7 @​@​ #
# \

Normal text\, a \link\ to nowhere\,\

#
# -\

a link to \var-copy\\,\

# +\

a link to \var-copy\\,\

#
# \

\htmlescp\\,\

#
# Looks like you failed 1 test of 1.

# Failed test 'test --htmldir and --htmlroot 3b' # at t/pod2html-lib.pl line 82. ===( 30;1 1/1 1/2 )==============================================Dubious\, test returned 1 (wstat 256\, 0x100) Failed 1/1 subtests ../ext/Pod-Html/t/yhtmldir3.t .. # --- htmldir3_expected.tmp 2014-01-01 21​:24​:00.046166879 +0100 # +++ /home/jkeenan/perl/ext/Pod-Html/t/htmldir3.html 2014-01-01 21​:24​:00.046166879 +0100 # @​@​ -24\,7 +24\,7 @​@​ #
# \

Normal text\, a \link\ to nowhere\,\

#
# -\

a link to \var-copy\\,\

# +\

a link to \var-copy\\,\

#
# \

\htmlescp\\,\

#

# Failed test 'test --htmldir and --htmlroot 3a' # at t/pod2html-lib.pl line 82.

not ok 2 - test --htmldir and --htmlroot 3a # --- htmldir3_expected.tmp 2014-01-01 21​:24​:00.064166871 +0100 # +++ /home/jkeenan/perl/ext/Pod-Html/t/htmldir3.html 2014-01-01 21​:24​:00.063166871 +0100 # @​@​ -24\,7 +24\,7 @​@​ #
# \

Normal text\, a \link\ to nowhere\,\

#
# -\

a link to \var-copy\\,\

# +\

a link to \var-copy\\,\

#
# \

\htmlescp\\,\

#
# Looks like you failed 2 tests of 2. Dubious\, test returned 2 (wstat 512\, 0x200) Failed 2/2 subtests

Test Summary Report

---

../ext/Pod-Html/t/cache.t (Wstat​: 512 Tests​: 3 Failed​: 1)   Failed test​: 3   Non-zero exit status​: 2   Parse errors​: Bad plan. You planned 10 tests but ran 3. ../ext/Pod-Html/t/crossref2.t (Wstat​: 256 Tests​: 1 Failed​: 1)   Failed test​: 1   Non-zero exit status​: 1 ../ext/Pod-Html/t/xhtmldir3.t (Wstat​: 256 Tests​: 1 Failed​: 1)   Failed test​: 1   Non-zero exit status​: 1 ../ext/Pod-Html/t/yhtmldir3.t (Wstat​: 512 Tests​: 2 Failed​: 2)   Failed tests​: 1-2   Non-zero exit status​: 2 Files=20\, Tests=31\, 1 wallclock secs ( 0.07 usr 0.02 sys + 2.38 cusr 0.30 csys = 2.77 CPU) Result​: FAIL /home/jkeenan/perl

2 more files are failing!

Repeat.

$ cd t;./perl harness -v ../ext/Pod-Html/t/*.t;cd - ===( 1;0 1/1 0/? 0/? 0/? 0/? 0/3 0/? 0/? )================ok ===( 2;0 1/10 0/? 0/? 0/? 0/3 0/? 0/? 0/? )==============Directory 'test.lib' exists (it shouldn't yet) - removing it at t/pod2html-lib.pl line 14. Directory 'test.lib' exists (it shouldn't yet) - removing it at t/pod2html-lib.pl line 14. t/feature2.t​: error opening ./pod2htmd.tmp for reading​: No such file or directory # Looks like your test exited with 2 before it could output anything. ===( 10;0 8/10 0/1 0/1 0/1 1/3 0/2 0/1 0/? )==============Dubious\, test returned 2 (wstat 512\, 0x200) Failed 2/2 subtests ===( 14;0 10/10 0/1 0/1 0/1 3/3 0/1 0/? )===================ok ===( 15;0 0/1 1/1 0/1 3/3 0/1 0/? 0/? 0/? )================ok ===( 15;0 0/1 1/1 0/1 0/1 0/? 0/? 0/? )=====================ok ===( 18;0 1/1 1/1 1/1 0/? 0/? 0/? 0/? 0/? )================ok ===( 18;0 1/1 1/1 0/? 0/? 0/? 0/? 0/? )=====================ok ===( 18;0 1/1 0/? 0/? 0/? 0/? 0/? 0/? )=====================ok ===( 19;1 1/2 0/? 0/? 0/? 0/? 0/? 0/? 0/? )================ok ===( 20;1 0/3 0/2 0/? 0/? 0/? 0/? 0/? )=====================Directory 'test.lib' exists (it shouldn't yet) - removing it at t/pod2html-lib.pl line 14. ===( 26;1 2/3 1/2 1/2 1/1 1/1 0/1 0/1 0/? )================ok ===( 27;1 2/3 1/2 2/2 1/1 0/1 0/1 0/? )=====================ok ===( 27;1 2/3 1/2 2/2 0/1 0/1 0/? 0/? )=====================ok ===( 28;1 2/3 2/2 0/1 0/1 0/? 0/? 0/? )=====================ok ===( 28;1 2/3 0/1 0/1 0/? 0/? 0/? 0/? )=====================ok ===( 31;1 1/1 1/1 0/? 0/? 0/? 0/? )==========================ok ===( 31;1 1/1 0/? 0/? 0/? 0/? )===============================ok ===( 32;1 1/1 0/? 0/? 0/? )====================================ok ===( 32;1 0/? 0/? 0/? )=========================================Directory 'test.lib' exists (it shouldn't yet) - removing it at t/pod2html-lib.pl line 14. ===( 33;1 1/1 0/? 0/2 )=========================================ok ===( 33;1 0/? 0/2 )============================================== # Failed test 'test --htmldir and --htmlroot 3b' # at t/pod2html-lib.pl line 82. ===( 34;1 1/? 0/2 )==============================================# --- htmldir3_expected.tmp 2014-01-01 21​:25​:01.240138807 +0100 # +++ /home/jkeenan/perl/ext/Pod-Html/t/htmldir3.html 2014-01-01 21​:25​:01.240138807 +0100 # @​@​ -24\,7 +24\,7 @​@​ #
# \

Normal text\, a \link\ to nowhere\,\

#
# -\

a link to \var-copy\\,\

# +\

a link to \var-copy\\,\

#
# \

\htmlescp\\,\

#
# Looks like you failed 1 test of 1.

# Failed test 'test --htmldir and --htmlroot 3b' # at t/pod2html-lib.pl line 82. ===( 35;1 1/1 1/2 )==============================================Dubious\, test returned 1 (wstat 256\, 0x100) Failed 1/1 subtests # --- htmldir3_expected.tmp 2014-01-01 21​:25​:01.245138803 +0100 # +++ /home/jkeenan/perl/ext/Pod-Html/t/htmldir3.html 2014-01-01 21​:25​:01.244138803 +0100 # @​@​ -24\,7 +24\,7 @​@​ #
# \

Normal text\, a \link\ to nowhere\,\

#
# -\

a link to \var-copy\\,\

# +\

a link to \var-copy\\,\

#
# \

\htmlescp\\,\

#
../ext/Pod-Html/t/yhtmldir3.t .. # Failed test 'test --htmldir and --htmlroot 3a' # at t/pod2html-lib.pl line 82.

not ok 2 - test --htmldir and --htmlroot 3a # --- htmldir3_expected.tmp 2014-01-01 21​:25​:01.261138798 +0100 # +++ /home/jkeenan/perl/ext/Pod-Html/t/htmldir3.html 2014-01-01 21​:25​:01.260138798 +0100 # @​@​ -24\,7 +24\,7 @​@​ #
# \

Normal text\, a \link\ to nowhere\,\

#
# -\

a link to \var-copy\\,\

# +\

a link to \var-copy\\,\

#
# \

\htmlescp\\,\

#
# Looks like you failed 2 tests of 2. Dubious\, test returned 2 (wstat 512\, 0x200) Failed 2/2 subtests

Test Summary Report

---

../ext/Pod-Html/t/feature2.t (Wstat​: 512 Tests​: 0 Failed​: 0)   Non-zero exit status​: 2   Parse errors​: Bad plan. You planned 2 tests but ran 0. ../ext/Pod-Html/t/xhtmldir3.t (Wstat​: 256 Tests​: 1 Failed​: 1)   Failed test​: 1   Non-zero exit status​: 1 ../ext/Pod-Html/t/yhtmldir3.t (Wstat​: 512 Tests​: 2 Failed​: 2)   Failed tests​: 1-2   Non-zero exit status​: 2 Files=20\, Tests=36\, 1 wallclock secs ( 0.06 usr 0.02 sys + 2.48 cusr 0.31 csys = 2.87 CPU) Result​: FAIL /home/jkeenan/perl

cache.t and crossref2.t are once again passing -- but now feature2.t is failing!

Repeat once more.

$ cd t;./perl harness -v ../ext/Pod-Html/t/*.t;cd - ===( 1;0 1/1 0/? 0/? 0/? 0/? 0/3 0/? 0/? )================ok ===( 1;0 0/? 0/? 0/? 0/? 0/3 0/? 0/? )=====================Directory 'test.lib' exists (it shouldn't yet) - removing it at t/pod2html-lib.pl line 14. ===( 3;1 0/? 0/? 1/1 1/1 0/3 0/? 0/1 0/? )================ok ===( 3;1 0/? 0/? 1/1 0/3 0/? 0/1 0/? )=====================ok ===( 5;1 1/10 0/? 0/3 0/? 1/1 0/? 0/? 0/? )==============ok ===( 10;1 5/10 0/1 1/3 0/2 0/? 0/? 0/? 0/? )==============ok ===( 12;1 5/10 0/1 0/2 0/? 0/? 0/? 0/? )===================ok ===( 18;1 0/1 1/2 0/? 0/? 0/? 0/? 0/? 0/? )================ok ===( 20;1 1/1 0/? 0/? 0/? 0/? 0/? 0/? 0/? )================ok ===( 21;1 1/2 0/? 0/? 0/? 0/? 0/? 0/? 0/? )================ok ===( 25;1 1/3 1/2 1/2 0/? 0/? 0/? 0/? 0/? )================ok ===( 28;1 3/3 1/2 0/? 0/? 0/? 0/? 0/? )=====================ok ===( 28;1 1/2 0/? 0/? 0/? 0/? 0/? 0/? )=====================ok ===( 30;1 0/1 1/1 0/? 0/1 0/1 0/? 0/? 0/? )================ok ===( 31;1 1/1 0/1 0/1 0/1 0/? 0/? 0/? )=====================ok ===( 32;1 0/1 0/1 1/1 0/? 0/? 0/? )==========================ok ===( 34;1 1/1 1/1 0/? 0/? 0/? )===============================ok ===( 34;1 1/1 0/? 0/? 0/? )====================================ok ===( 34;1 0/? 0/? 0/? )=========================================Directory 'test.lib' exists (it shouldn't yet) - removing it at t/pod2html-lib.pl line 14.

# Failed test 'test --htmldir and --htmlroot 3b' # at t/pod2html-lib.pl line 82. ===( 35;1 0/1 1/? 0/2 )=========================================# --- htmldir3_expected.tmp 2014-01-01 21​:26​:10.249107428 +0100 # +++ /home/jkeenan/perl/ext/Pod-Html/t/htmldir3.html 2014-01-01 21​:26​:10.248107429 +0100 # @​@​ -24\,7 +24\,7 @​@​ #
# \

Normal text\, a \link\ to nowhere\,\

#
# -\

a link to \var-copy\\,\

# +\

a link to \var-copy\\,\

#
# \

\htmlescp\\,\

#
# Looks like you failed 1 test of 1.

# Failed test 'test --htmldir and --htmlroot 3b' # at t/pod2html-lib.pl line 82. ===( 37;1 1/1 1/1 1/2 )=========================================# --- htmldir3_expected.tmp 2014-01-01 21​:26​:10.252107427 +0100 # +++ /home/jkeenan/perl/ext/Pod-Html/t/htmldir3.html 2014-01-01 21​:26​:10.252107426 +0100 # @​@​ -24\,7 +24\,7 @​@​ #
# \

Normal text\, a \link\ to nowhere\,\

#
# -\

a link to \var-copy\\,\

# +\

a link to \var-copy\\,\

#
# \

\htmlescp\\,\

#
Dubious\, test returned 1 (wstat 256\, 0x100) Failed 1/1 subtests ===( 37;1 1/1 1/2 )==============================================ok ../ext/Pod-Html/t/yhtmldir3.t .. # Failed test 'test --htmldir and --htmlroot 3a' # at t/pod2html-lib.pl line 82.

not ok 2 - test --htmldir and --htmlroot 3a # --- htmldir3_expected.tmp 2014-01-01 21​:26​:10.269107418 +0100 # +++ /home/jkeenan/perl/ext/Pod-Html/t/htmldir3.html 2014-01-01 21​:26​:10.268107419 +0100 # @​@​ -24\,7 +24\,7 @​@​ #
# \

Normal text\, a \link\ to nowhere\,\

#
# -\

a link to \var-copy\\,\

# +\

a link to \var-copy\\,\

#
# \

\htmlescp\\,\

#
# Looks like you failed 2 tests of 2. Dubious\, test returned 2 (wstat 512\, 0x200) Failed 2/2 subtests

Test Summary Report

---

../ext/Pod-Html/t/xhtmldir3.t (Wstat​: 256 Tests​: 1 Failed​: 1)   Failed test​: 1   Non-zero exit status​: 1 ../ext/Pod-Html/t/yhtmldir3.t (Wstat​: 512 Tests​: 2 Failed​: 2)   Failed tests​: 1-2   Non-zero exit status​: 2 Files=20\, Tests=38\, 1 wallclock secs ( 0.07 usr 0.01 sys + 2.08 cusr 0.26 csys = 2.42 CPU) Result​: FAIL /home/jkeenan/perl

Now we're back to just x/y htmldir3.t failing!

Further repetitions vary. Often 3 tests fail​: x/y and one of cache\, crossref2\, feature2.

[p5pRT]

From @jkeenan

On Wed Jan 01 12​:35​:53 2014\, jkeenan wrote​:

Two calendar years later ... and this just got weirder!

I created a branch on the dromedary test server today and added the two test files (with minor updatings) originally posted in this ticket.

I then configure\, built and tested perl in that branch. As expected\, the two additional files failed.

I then repeatedly ran this set of commands​:

##### cd t;./perl harness -v ../ext/Pod-Html/t/*.t; cd- #####

... and got different results each time!

t/xhtmldir3.t and t/yhtmldir3.t failed each time\, as expected. But each repetition generated failures in 0 to 2 *other*\, previously passing files! The output is in the attachment.

More data. Tonight I built blead on the dromedary server. All tests\, including those in ext/Pod-Html/t/\, passed. (The command I run there is​: TEST_JOBS=${TEST_JOBS} make -j${TEST_JOBS} test_harness ... with TEST_JOBS=8.)

I then ran those same tests with the harness 8 times -- and got 8 different results.

##### cd t;./perl harness -v ../ext/Pod-Html/t/*.t; cd- #####

Results attached.

It is interesting to note that many\, but not all\, of the test failures in the attachment are failures like these two​:

##### # -\

\var-copy\\

# +\

\var-copy\\

#
# -\

\"$"" in var-copy\\

# +\

\"$"" in var-copy\\

#####

That is\, the output expected -- the '-' lines in the diff excerpt above -- is an \ tag where something like the following has been interpolated in​:

##### href="../testdir/test.lib/var-copy.html #####

But what is actually generated is simply​: \

In my attempt to make Pod-Html CPANnable this type of test failure is one I'm getting in many files -- but at least I'm getting them consistently. And it's the only type of failure I'm still getting. So\, while there may well be errors in my refactoring of the code\, the fundamental problem may have been there all along but simply have been masked.

Thank you very much. Jim Keenan

From @jkeenan

$ cd t;./perl harness -v ../ext/Pod-Html/t/*t; cd - Directory 'test.lib' exists (it shouldn't yet) - removing it at t/pod2html-lib.pl line 14. ===( 1;0 1/1 0/? 0/? 0/1 0/1 0/3 0/? 0/? )================ok ===( 4;0 2/10 0/? 1/1 0/1 0/3 0/? 0/1 0/? )==============ok ===( 4;0 2/10 0/? 0/1 0/3 0/? 0/1 0/? )=================== # Failed test 'podpath' # at t/cache.t line 38. # got​: 't' # expected​: 'scooby​:shaggy​:fred​:velma​:daphne' ===( 7;0 3/10 0/1 1/1 0/3 0/? 1/1 0/? 0/? )==============ok ===( 7;0 3/10 0/1 0/3 0/? 1/1 0/? 0/? )===================ok ===( 7;0 3/10 0/1 0/3 0/2 0/? 0/? 0/? )===================# Looks like you failed 1 test of 10. ===( 9;0 4/10 0/1 1/3 0/2 0/? 0/? 0/? 0/? )==============ok ===( 17;0 10/10 0/1 0/2 0/? 0/? 0/? 0/? )===================Dubious\, test returned 1 (wstat 256\, 0x100) Failed 1/10 subtests ===( 17;0 0/1 0/2 0/? 0/? 0/? 0/? 0/? )===================== # Failed test 'cross references' # at t/pod2html-lib.pl line 82. ===( 18;0 1/1 0/2 0/? 0/? 0/? 0/? 0/? 0/? )================# --- crossref_expected.tmp 2014-01-08 02​:58​:30.186060121 +0100 # +++ /home/jkeenan/perl/ext/Pod-Html/t/crossref.html 2014-01-08 02​:58​:30.185060124 +0100 # @​@​ -35\,15 +35\,15 @​@​ #
# \

\"non existant section"\\

#
# -\

\var-copy\\

# +\

\var-copy\\

#
# -\

\"$"" in var-copy\\

# +\

\"$"" in var-copy\\

#
# \

\var-copy\\

#
# \

\var-copy/$"\\

#
# -\

\"First​:" in podspec-copy\\

# +\

\"First​:" in podspec-copy\\

#
# \

\podspec-copy/First​:\\

#
# Looks like you failed 1 test of 1. ===( 19;0 1/1 1/2 0/? 0/? 0/? 0/? 0/? 0/? )================ok ===( 20;0 1/1 0/? 0/? 0/? 0/? 0/? 0/? )=====================Dubious\, test returned 1 (wstat 256\, 0x100) Failed 1/1 subtests ===( 23;0 2/2 1/3 0/? 0/? 0/? 0/? 0/? 0/? )================ok ===( 23;0 1/3 0/? 0/? 0/? 0/? 0/? 0/? )=====================ok ===( 25;0 0/? 0/? 0/? 0/? 0/? 0/? 0/? )=====================Directory 'test.lib' exists (it shouldn't yet) - removing it at t/pod2html-lib.pl line 14. ===( 26;0 0/2 0/2 0/1 1/1 0/? 0/1 0/? 0/? )================ok ===( 29;0 0/2 1/2 1/1 0/1 1/1 0/? 0/? )=====================ok ===( 30;0 1/2 1/2 1/1 0/1 0/? 0/? )==========================ok ===( 30;0 1/2 1/2 0/1 0/? 0/? )===============================ok ===( 31;0 1/2 0/1 0/? 0/? )====================================ok ===( 33;0 1/1 0/? 0/? )=========================================ok ===( 34;0 1/1 0/? )==============================================ok ../ext/Pod-Html/t/podnoerr.t ... ok 1 - pod error section ok

Test Summary Report

---

../ext/Pod-Html/t/cache.t (Wstat​: 256 Tests​: 10 Failed​: 1)   Failed test​: 4   Non-zero exit status​: 1 ../ext/Pod-Html/t/crossref2.t (Wstat​: 256 Tests​: 1 Failed​: 1)   Failed test​: 1   Non-zero exit status​: 1 Files=18\, Tests=35\, 0 wallclock secs ( 0.06 usr 0.02 sys + 1.72 cusr 0.20 csys = 2.00 CPU) Result​: FAIL /home/jkeenan/perl [dromedary-001​:perl] 1022 $ cd t;./perl harness -v ../ext/Pod-Html/t/*t; cd - Directory 'test.lib' exists (it shouldn't yet) - removing it at t/pod2html-lib.pl line 14. ===( 2;0 1/1 0/? 0/? 1/1 0/1 0/3 0/? 0/1 )================ok ===( 2;0 1/1 0/? 0/? 0/1 0/3 0/? 0/1 )=====================ok ===( 3;0 0/? 0/? 1/1 0/3 0/? 0/1 0/? 0/? )================ok ===( 4;0 0/? 0/? 0/3 0/? 1/1 0/? 0/? 0/? )================ok ===( 5;0 1/10 0/? 0/3 0/? 0/? 0/? 0/? 0/? )============== # Failed test 'podpath' # at t/cache.t line 38. # got​: '' # expected​: 'scooby​:shaggy​:fred​:velma​:daphne'

# Failed test 'podroot' # at t/cache.t line 39. # got​: '.' # expected​: '/home/jkeenan/perl/ext/Pod-Html' ===( 10;0 5/10 0/1 1/3 0/2 0/? 0/? 0/? 0/? )==============ok ===( 13;0 5/10 1/1 0/2 0/? 0/? 0/? 0/? 0/? )==============# Looks like you failed 2 tests of 10.

# Failed test 'misc pod-html --verbose warnings' # at t/feature2.t line 27. # 'scanning for directory cache # scanning for directory cache # caching directories for later use # Converting input file /home/jkeenan/perl/ext/Pod-Html/t/feature2.pod # ' # doesn't match '(?^x​: # \Acaching\ directories\ for\ later\ use\n # Converting\ input\ file\ \S+[/\\\]]feature2\.pod\n\z # )' # Looks like you failed 1 test of 2. ok ===( 18;0 10/10 0/2 0/? 0/? 0/? 0/? 0/? )===================Dubious\, test returned 2 (wstat 512\, 0x200) Failed 2/10 subtests ===( 19;0 1/2 0/? 0/? 0/? 0/? 0/? 0/? 0/? )================Dubious\, test returned 1 (wstat 256\, 0x100) Failed 1/2 subtests ===( 20;0 0/? 0/? 0/? 0/? 0/? 0/? 0/? )=====================Directory 'test.lib' exists (it shouldn't yet) - removing it at t/pod2html-lib.pl line 14.

# Failed test 'test --htmldir and --htmlroot 1a' # at t/pod2html-lib.pl line 82. ===( 21;0 1/2 0/3 0/2 0/? 0/? 0/? 0/? 0/? )================# --- htmldir1_expected.tmp 2014-01-08 02​:58​:54.960990762 +0100 # +++ /home/jkeenan/perl/ext/Pod-Html/t/htmldir1.html 2014-01-08 02​:58​:54.960990762 +0100 # @​@​ -26\,7 +26\,7 @​@​ #
# \

Normal text\, a \link\ to nowhere\,\

#
# -\

a link to \var-copy\\,\

# +\

a link to \var-copy\\,\

#
# \

\htmlescp\\,\

#
===( 24;0 1/2 2/3 1/2 0/? 0/? 0/? 0/? 0/? )================# Looks like you failed 1 test of 2. Dubious\, test returned 1 (wstat 256\, 0x100) Failed 1/2 subtests ===( 25;0 2/3 1/2 0/2 0/? 0/? 0/? 0/? )=====================ok ===( 28;0 2/2 1/2 0/1 0/1 0/? 0/? 0/? 0/? )================ok ===( 28;0 1/2 0/1 0/1 0/? 0/? 0/? 0/? )===================== # Failed test 'test --htmldir and --htmlroot 5' # at t/pod2html-lib.pl line 82. ===( 29;0 1/2 1/1 0/1 0/1 0/1 0/? 0/? )=====================# --- htmldir5_expected.tmp 2014-01-08 02​:58​:54.991990675 +0100 # +++ /home/jkeenan/perl/ext/Pod-Html/t/htmldir5.html 2014-01-08 02​:58​:54.990990678 +0100 # @​@​ -24\,7 +24\,7 @​@​ #
# \

Normal text\, a \link\ to nowhere\,\

#
# -\

a link to \var-copy\\,\

# +\

a link to \var-copy\\,\

#
# \

\htmlescp\\,\

#
# Looks like you failed 1 test of 1. ===( 30;0 1/2 1/1 1/1 0/1 0/1 0/? 0/? )=====================Dubious\, test returned 1 (wstat 256\, 0x100) Failed 1/1 subtests ===( 30;0 1/2 1/1 0/1 0/1 0/? 0/? )==========================ok ===( 30;0 1/2 0/1 0/1 0/? 0/? )===============================ok ===( 33;1 1/1 1/1 0/? 0/? )====================================ok ===( 33;1 1/1 0/? 0/? )=========================================ok ===( 35;1 1/1 1/1 )==============================================ok ../ext/Pod-Html/t/podnoerr.t ... ok

Test Summary Report

---

../ext/Pod-Html/t/cache.t (Wstat​: 512 Tests​: 10 Failed​: 2)   Failed tests​: 4-5   Non-zero exit status​: 2 ../ext/Pod-Html/t/feature2.t (Wstat​: 256 Tests​: 2 Failed​: 1)   Failed test​: 2   Non-zero exit status​: 1 ../ext/Pod-Html/t/htmldir1.t (Wstat​: 256 Tests​: 2 Failed​: 1)   Failed test​: 1   Non-zero exit status​: 1 ../ext/Pod-Html/t/htmldir5.t (Wstat​: 256 Tests​: 1 Failed​: 1)   Failed test​: 1   Non-zero exit status​: 1 Files=18\, Tests=35\, 1 wallclock secs ( 0.05 usr 0.01 sys + 1.68 cusr 0.21 csys = 1.95 CPU) Result​: FAIL /home/jkeenan/perl [dromedary-001​:perl] 1022 $ cd t;./perl harness -v ../ext/Pod-Html/t/*t; cd - ===( 1;0 1/1 0/? 0/? 0/? 0/? 0/3 0/? 0/? )================ok ===( 2;0 1/10 0/? 0/? 0/? 0/3 0/? 0/? 0/? )==============Directory 'test.lib' exists (it shouldn't yet) - removing it at t/pod2html-lib.pl line 14. Directory 'test.lib' exists (it shouldn't yet) - removing it at t/pod2html-lib.pl line 14. ok ===( 16;0 0/1 1/1 0/1 3/3 1/2 0/1 0/? 0/? )================ok ===( 18;0 1/1 1/1 0/1 3/3 0/1 0/? 0/? 0/? )================ok ===( 18;0 1/1 0/1 3/3 0/1 0/? 0/? 0/? )=====================ok ===( 19;0 1/1 3/3 0/1 0/? 0/? 0/? 0/? 0/? )================ok ===( 19;0 1/1 0/1 0/? 0/? 0/? 0/? 0/? )=====================ok ===( 20;0 1/1 0/? 0/? 0/? 0/? 0/? 0/? 0/? )================ok ===( 21;0 1/2 0/? 0/? 0/? 0/? 0/? 0/? 0/? )================ok ===( 22;0 0/3 0/? 0/? 0/? 0/? 0/? 0/? )=====================Directory 'test.lib' exists (it shouldn't yet) - removing it at t/pod2html-lib.pl line 14. ===( 26;0 2/3 1/2 0/2 0/1 1/1 0/1 0/1 0/? )================ok ===( 28;0 2/3 1/2 1/2 1/1 0/1 0/1 0/? 0/? )================ok ===( 28;0 2/3 1/2 1/2 0/1 0/1 0/? 0/? )=====================ok ===( 31;0 3/3 1/2 1/1 0/1 0/? 0/? )==========================ok ===( 31;0 1/2 1/1 0/1 0/? 0/? )===============================ok ===( 32;0 1/2 1/1 0/? 0/? )====================================ok ===( 32;0 1/2 0/? 0/? )=========================================ok ===( 34;0 1/1 0/? )==============================================ok ../ext/Pod-Html/t/podnoerr.t ... ok 1 - pod error section ok All tests successful. Files=18\, Tests=35\, 0 wallclock secs ( 0.05 usr 0.01 sys + 1.48 cusr 0.18 csys = 1.72 CPU) Result​: PASS /home/jkeenan/perl [dromedary-001​:perl] 1022 $ cd t;./perl harness -v ../ext/Pod-Html/t/*t; cd - ===( 1;0 1/1 0/? 0/? 0/? 0/? 0/3 0/? 0/? )================ok ===( 1;0 0/? 0/? 0/? 0/? 0/3 0/? 0/? )=====================Directory 'test.lib' exists (it shouldn't yet) - removing it at t/pod2html-lib.pl line 14. Directory 'test.lib' exists (it shouldn't yet) - removing it at t/pod2html-lib.pl line 14. ===( 2;0 1/10 0/1 0/1 0/1 0/3 0/? 0/? 0/? )============== # Failed test 'podpath' # at t/cache.t line 38. # got​: '' # expected​: 'scooby​:shaggy​:fred​:velma​:daphne'

# Failed test 'podroot' # at t/cache.t line 39. # got​: '.' # expected​: '/home/jkeenan/perl/ext/Pod-Html'

# Failed test 'cross references' # at t/pod2html-lib.pl line 82. ===( 7;0 5/10 0/1 1/1 0/1 0/3 0/? 0/1 0/? )==============# --- crossref_expected.tmp 2014-01-08 02​:59​:14.307936532 +0100 # +++ /home/jkeenan/perl/ext/Pod-Html/t/crossref.html 2014-01-08 02​:59​:14.307936532 +0100 # @​@​ -35\,15 +35\,15 @​@​ #
# \

\"non existant section"\\

#
# -\

\var-copy\\

# +\

\var-copy\\

#
# -\

\"$"" in var-copy\\

# +\

\"$"" in var-copy\\

#
# \

\var-copy\\

#
# \

\var-copy/$"\\

#
# -\

\"First​:" in podspec-copy\\

# +\

\"First​:" in podspec-copy\\

#
# \

\podspec-copy/First​:\\

#
# Looks like you failed 1 test of 1. ===( 8;0 5/10 0/1 1/1 0/1 1/3 0/2 0/1 0/? )==============ok ===( 10;0 5/10 0/1 1/1 0/1 0/2 0/1 0/? )===================# Looks like you failed 2 tests of 10. Dubious\, test returned 1 (wstat 256\, 0x100) Failed 1/1 subtests ===( 15;0 7/10 1/1 1/1 0/2 1/1 0/? 0/? 0/? )==============ok ===( 16;0 8/10 1/1 0/2 1/1 0/? 0/? 0/? )===================ok ===( 17;0 9/10 0/2 1/1 0/? 0/? 0/? 0/? )===================ok ===( 18;0 10/10 0/2 0/? 0/? 0/? 0/? 0/? )===================Dubious\, test returned 2 (wstat 512\, 0x200) Failed 2/10 subtests ===( 19;0 1/2 0/? 0/? 0/? 0/? 0/? 0/? 0/? )================ok ===( 21;0 1/2 0/? 0/? 0/? 0/? 0/? 0/? 0/? )================Directory 'test.lib' exists (it shouldn't yet) - removing it at t/pod2html-lib.pl line 14. Directory 'test.lib' exists (it shouldn't yet) - removing it at t/pod2html-lib.pl line 14. Error removing temporary directory 'testdir/test.lib' at t/pod2html-lib.pl line 27. ===( 23;0 2/2 0/3 1/2 0/2 0/1 0/1 0/? 0/? )================ok ===( 27;0 2/3 1/2 1/2 0/1 1/1 0/1 0/? 0/? )================ok ===( 28;0 2/3 2/2 1/2 0/1 0/1 0/1 0/? )=====================ok ===( 29;0 2/3 1/2 1/1 0/1 0/1 0/? 0/? )=====================ok ===( 29;0 2/3 1/2 0/1 0/1 0/? 0/? )==========================ok ===( 31;0 2/2 0/1 0/1 0/? 0/? )===============================ok ===( 32;0 1/1 0/1 0/? 0/? )====================================ok ===( 33;0 1/1 0/? 0/? )=========================================ok ===( 35;0 1/1 1/1 )==============================================ok ../ext/Pod-Html/t/poderr.t ..... ok

Test Summary Report

---

../ext/Pod-Html/t/crossref3.t (Wstat​: 256 Tests​: 1 Failed​: 1)   Failed test​: 1   Non-zero exit status​: 1 ../ext/Pod-Html/t/cache.t (Wstat​: 512 Tests​: 10 Failed​: 2)   Failed tests​: 4-5   Non-zero exit status​: 2 Files=18\, Tests=35\, 0 wallclock secs ( 0.06 usr 0.02 sys + 1.53 cusr 0.22 csys = 1.83 CPU) Result​: FAIL /home/jkeenan/perl [dromedary-001​:perl] 1022 $ cd t;./perl harness -v ../ext/Pod-Html/t/*t; cd - ===( 1;0 1/1 0/? 0/? 0/? 0/? 0/3 0/? 0/? )================Directory 'test.lib' exists (it shouldn't yet) - removing it at t/pod2html-lib.pl line 14. cannot remove directory for testdir/test.lib​: Directory not empty at t/pod2html-lib.pl line 27. ok ===( 2;0 0/? 0/? 1/1 0/1 0/3 0/? 0/? 0/? )================ok ===( 3;0 1/10 0/? 0/1 0/3 0/? 0/1 0/? 0/? )============== # Failed test 'cross references' # at t/pod2html-lib.pl line 82. ===( 5;0 2/10 0/? 1/1 0/3 0/? 0/1 0/? 0/? )==============# --- crossref_expected.tmp 2014-01-08 02​:59​:24.314908458 +0100 # +++ /home/jkeenan/perl/ext/Pod-Html/t/crossref.html 2014-01-08 02​:59​:24.313908461 +0100 # @​@​ -35\,15 +35\,15 @​@​ #
# \

\"non existant section"\\

#
# -\

\var-copy\\

# +\

\var-copy\\

#
# -\

\"$"" in var-copy\\

# +\

\"$"" in var-copy\\

#
# \

\var-copy\\

#
# \

\var-copy/$"\\

#
# -\

\"First​:" in podspec-copy\\

# +\

\"First​:" in podspec-copy\\

#
# \

\podspec-copy/First​:\\

#
# Looks like you failed 1 test of 1. Dubious\, test returned 1 (wstat 256\, 0x100) Failed 1/1 subtests ===( 12;0 7/10 0/1 1/3 0/2 1/1 0/? 0/? 0/? )==============ok ===( 12;0 7/10 0/1 1/3 0/2 0/? 0/? 0/? )===================ok ===( 17;0 10/10 0/1 0/2 0/? 0/? 0/? 0/? )===================ok ===( 18;0 0/1 1/2 0/? 0/? 0/? 0/? 0/? 0/? )================ok ===( 20;0 1/1 0/? 0/? 0/? 0/? 0/? 0/? 0/? )================ok ===( 22;0 1/2 1/3 0/? 0/? 0/? 0/? 0/? 0/? )================ok ===( 24;0 2/3 0/? 0/? 0/? 0/? 0/? 0/? )=====================ok ===( 26;0 0/2 0/2 0/? 1/1 0/? 0/? 0/? 0/? )================Directory 'test.lib' exists (it shouldn't yet) - removing it at t/pod2html-lib.pl line 14. ok ===( 29;0 1/2 1/2 1/1 0/1 0/1 0/? 0/? )=====================ok ===( 30;0 2/2 1/2 0/1 0/1 0/? 0/? )==========================ok ===( 31;0 2/2 0/1 0/1 0/? 0/? )===============================ok ===( 32;0 1/1 0/1 0/? 0/? )====================================ok ===( 33;0 1/1 0/? 0/? )=========================================ok ===( 35;0 1/1 1/1 )==============================================ok ../ext/Pod-Html/t/poderr.t ..... ok

Test Summary Report

---

../ext/Pod-Html/t/crossref.t (Wstat​: 256 Tests​: 1 Failed​: 1)   Failed test​: 1   Non-zero exit status​: 1 Files=18\, Tests=35\, 0 wallclock secs ( 0.05 usr 0.02 sys + 1.67 cusr 0.21 csys = 1.95 CPU) Result​: FAIL /home/jkeenan/perl [dromedary-001​:perl] 1022 $ cd t;./perl harness -v ../ext/Pod-Html/t/*t; cd - Directory 'test.lib' exists (it shouldn't yet) - removing it at t/pod2html-lib.pl line 14. ===( 1;0 1/1 0/? 0/? 0/1 0/1 0/3 0/? 0/? )================ok ===( 4;0 2/10 0/? 1/1 0/1 0/3 0/? 0/1 0/? )==============ok ===( 4;0 2/10 0/1 0/1 0/3 0/? 0/1 0/? )=================== # Failed test 'podpath' # at t/cache.t line 38. # got​: 't' # expected​: 'scooby​:shaggy​:fred​:velma​:daphne' ===( 6;0 3/10 0/1 1/1 0/3 0/? 0/1 0/? 0/? )==============ok ===( 7;0 3/10 0/1 0/3 0/? 1/1 0/? 0/? 0/? )==============ok ===( 8;0 3/10 0/1 1/3 0/2 0/? 0/? 0/? 0/? )==============ok ===( 10;0 3/10 0/1 0/2 0/? 0/? 0/? 0/? )===================# Looks like you failed 1 test of 10. ===( 14;0 6/10 1/1 0/2 0/? 0/? 0/? 0/? 0/? )==============Dubious\, test returned 1 (wstat 256\, 0x100) Failed 1/10 subtests ===( 18;0 1/1 0/2 0/? 0/? 0/? 0/? 0/? )=====================ok ===( 19;0 1/2 0/? 0/? 0/? 0/? 0/? 0/? 0/? )================ok ===( 20;0 0/? 0/? 0/? 0/? 0/? 0/? 0/? )=====================Directory 'test.lib' exists (it shouldn't yet) - removing it at t/pod2html-lib.pl line 14. ===( 21;0 1/2 0/3 0/2 0/2 0/? 0/? 0/? 0/? )================Directory 'test.lib' exists (it shouldn't yet) - removing it at t/pod2html-lib.pl line 14. ===( 24;0 2/2 1/3 1/2 0/2 0/1 0/1 0/1 0/? )================ok ===( 28;0 2/3 1/2 1/2 1/1 1/1 0/1 0/? 0/? )================ok ===( 28;0 2/3 1/2 1/2 1/1 0/1 0/? 0/? )=====================ok ===( 31;0 3/3 2/2 1/2 1/1 0/1 0/? 0/? )=====================ok ===( 31;0 2/2 1/2 1/1 0/1 0/? 0/? )==========================ok ===( 31;0 1/2 1/1 0/1 0/? 0/? )===============================ok ===( 31;0 1/2 0/1 0/? 0/? )====================================ok ===( 33;0 1/1 0/? 0/? )=========================================ok ===( 35;0 1/1 1/1 )==============================================ok ../ext/Pod-Html/t/podnoerr.t ... ok

Test Summary Report

---

../ext/Pod-Html/t/cache.t (Wstat​: 256 Tests​: 10 Failed​: 1)   Failed test​: 4   Non-zero exit status​: 1 Files=18\, Tests=35\, 0 wallclock secs ( 0.05 usr 0.02 sys + 1.60 cusr 0.21 csys = 1.88 CPU) Result​: FAIL /home/jkeenan/perl [dromedary-001​:perl] 1022 $ cd t;./perl harness -v ../ext/Pod-Html/t/*t; cd - ===( 1;0 1/1 0/? 0/? 0/? 0/? 0/3 0/? 0/? )================ok ===( 2;0 1/10 0/? 0/? 0/? 0/3 0/? 0/? 0/? )==============Directory 'test.lib' exists (it shouldn't yet) - removing it at t/pod2html-lib.pl line 14. Directory 'test.lib' exists (it shouldn't yet) - removing it at t/pod2html-lib.pl line 14. ok ===( 13;0 1/1 0/1 0/1 1/3 0/2 0/? 0/? 0/? )================ok ===( 14;0 0/1 0/1 2/3 0/2 0/1 0/? 0/? )===================== # Failed test 'misc pod-html --verbose warnings' # at t/feature2.t line 27. # 'scanning for directory cache # scanning for directory cache # caching directories for later use # Converting input file /home/jkeenan/perl/ext/Pod-Html/t/feature2.pod # ' # doesn't match '(?^x​: # \Acaching\ directories\ for\ later\ use\n # Converting\ input\ file\ \S+[/\\\]]feature2\.pod\n\z # )' # Looks like you failed 1 test of 2. cannot open /home/jkeenan/perl/ext/Pod-Html/t/crossref.html​: No such file or directory at t/pod2html-lib.pl line 69\, \ chunk 1. # Looks like your test exited with 2 before it could output anything. ===( 15;0 0/1 1/1 2/3 0/2 0/1 0/? 0/? 0/? )================ok ===( 16;0 0/1 1/1 0/2 0/1 0/? 0/? 0/? )=====================Dubious\, test returned 2 (wstat 512\, 0x200) Failed 1/1 subtests ===( 16;0 1/1 0/2 0/1 0/? 0/? 0/? 0/? )=====================ok ===( 18;0 1/2 1/1 0/? 0/? 0/? 0/? 0/? 0/? )================Dubious\, test returned 1 (wstat 256\, 0x100) Failed 1/2 subtests ===( 19;0 1/1 0/? 0/? 0/? 0/? 0/? 0/? )=====================ok ===( 20;0 1/2 0/? 0/? 0/? 0/? 0/? 0/? 0/? )================ok ===( 21;0 0/3 0/? 0/? 0/? 0/? 0/? 0/? )=====================Directory 'test.lib' exists (it shouldn't yet) - removing it at t/pod2html-lib.pl line 14. ===( 26;0 2/3 1/2 1/2 0/1 1/1 0/1 0/? 0/? )================ok ===( 27;0 2/3 1/2 1/2 1/1 0/1 0/? 0/? 0/? )================ok ===( 27;0 2/3 1/2 1/2 0/1 0/1 0/? 0/? )=====================ok ===( 30;0 3/3 2/2 0/1 0/1 0/? 0/? )==========================ok ===( 30;0 3/3 0/1 0/1 0/? 0/? )===============================ok ===( 31;0 1/1 0/1 0/? 0/? )====================================ok ===( 32;0 1/1 0/? 0/? )=========================================ok ===( 33;0 1/1 0/? )==============================================ok ../ext/Pod-Html/t/podnoerr.t ... ok 1 - pod error section ok

Test Summary Report

---

../ext/Pod-Html/t/crossref3.t (Wstat​: 512 Tests​: 0 Failed​: 0)   Non-zero exit status​: 2   Parse errors​: Bad plan. You planned 1 tests but ran 0. ../ext/Pod-Html/t/feature2.t (Wstat​: 256 Tests​: 2 Failed​: 1)   Failed test​: 2   Non-zero exit status​: 1 Files=18\, Tests=34\, 0 wallclock secs ( 0.05 usr 0.02 sys + 1.47 cusr 0.18 csys = 1.72 CPU) Result​: FAIL /home/jkeenan/perl [dromedary-001​:perl] 1022 $ cd t;./perl harness -v ../ext/Pod-Html/t/*t; cd - Directory 'test.lib' exists (it shouldn't yet) - removing it at t/pod2html-lib.pl line 14. ===( 1;0 1/1 0/? 0/? 0/1 0/1 0/3 0/? 0/1 )================ok ===( 1;0 0/? 0/? 0/1 0/1 0/3 0/? 0/1 )=====================cannot remove directory for testdir/test.lib​: No such file or directory at t/pod2html-lib.pl line 27. ===( 3;0 0/? 0/? 1/1 1/1 0/3 0/? 0/1 0/? )================ok ===( 3;0 0/? 0/? 1/1 0/3 0/? 0/1 0/? )=====================ok ===( 4;0 0/10 0/? 0/3 0/? 1/1 0/? 0/? 0/? )==============ok ===( 10;0 5/10 0/1 1/3 0/2 0/? 0/? 0/? 0/? )==============ok ===( 13;0 6/10 0/1 0/2 0/? 0/? 0/? 0/? )===================ok ===( 18;0 1/1 0/2 0/? 0/? 0/? 0/? 0/? 0/? )================ok ===( 19;0 1/2 0/? 0/? 0/? 0/? 0/? 0/? 0/? )================ok ===( 20;0 0/? 0/? 0/? 0/? 0/? 0/? 0/? )=====================t/htmldir4.t​: error opening ./pod2htmd.tmp for reading​: No such file or directory # Looks like your test exited with 2 before it could output anything. ===( 21;0 0/? 1/3 0/2 0/2 0/? 0/? 0/? 0/? )================Dubious\, test returned 2 (wstat 512\, 0x200) Failed 2/2 subtests ===( 22;0 0/? 1/3 1/2 0/? 0/? 0/? 0/? 0/? )================ok ===( 24;0 0/? 2/3 0/? 0/? 0/? 0/? 0/? )=====================ok ===( 26;0 1/2 0/? 0/? 0/? 0/? 0/? 0/? )=====================Directory 'test.lib' exists (it shouldn't yet) - removing it at t/pod2html-lib.pl line 14. ===( 28;0 2/2 0/1 1/1 0/1 0/1 0/? 0/? )=====================ok ===( 28;0 0/1 1/1 0/1 0/1 0/? 0/? )==========================ok ===( 29;0 1/1 0/1 0/1 0/? 0/? )===============================ok ===( 30;0 0/1 1/1 0/? 0/? )====================================ok ===( 31;0 1/1 0/? 0/? )=========================================ok ===( 32;0 1/1 0/? )==============================================ok ../ext/Pod-Html/t/podnoerr.t ... ok 1 - pod error section ok

Test Summary Report

---

../ext/Pod-Html/t/htmldir4.t (Wstat​: 512 Tests​: 0 Failed​: 0)   Non-zero exit status​: 2   Parse errors​: Bad plan. You planned 2 tests but ran 0. Files=18\, Tests=33\, 0 wallclock secs ( 0.04 usr 0.02 sys + 1.85 cusr 0.20 csys = 2.11 CPU) Result​: FAIL /home/jkeenan/perl [dromedary-001​:perl] 1022 $ cd t;./perl harness -v ../ext/Pod-Html/t/*t; cd - ===( 1;0 1/1 0/? 0/? 0/? 0/? 0/3 0/? 0/? )================ok ===( 2;0 1/10 0/? 0/? 0/? 0/3 0/? 0/? 0/? )==============Directory 'test.lib' exists (it shouldn't yet) - removing it at t/pod2html-lib.pl line 14.

# Failed test 'podpath' # at t/cache.t line 38. # got​: '' # expected​: 'scooby​:shaggy​:fred​:velma​:daphne'

# Failed test 'podroot' # at t/cache.t line 39. # got​: '.' # expected​: '/home/jkeenan/perl/ext/Pod-Html' Directory 'test.lib' exists (it shouldn't yet) - removing it at t/pod2html-lib.pl line 14. ===( 7;0 5/10 0/1 0/1 0/1 1/3 0/? 0/? 0/? )==============ok ===( 10;0 6/10 0/1 0/1 0/1 0/2 0/? 0/? )===================# Looks like you failed 2 tests of 10. Dubious\, test returned 2 (wstat 512\, 0x200) Failed 2/10 subtests ===( 15;0 1/1 0/1 0/1 0/2 0/? 0/? 0/? 0/? )================ok ===( 15;0 0/1 0/1 0/2 0/1 0/? 0/? 0/? )===================== # Failed test 'misc pod-html --verbose warnings' # at t/feature2.t line 27. # 'scanning for directory cache # scanning for directory cache # caching directories for later use # Converting input file /home/jkeenan/perl/ext/Pod-Html/t/feature2.pod # ' # doesn't match '(?^x​: # \Acaching\ directories\ for\ later\ use\n # Converting\ input\ file\ \S+[/\\\]]feature2\.pod\n\z # )' # Looks like you failed 1 test of 2. ===( 16;0 1/1 0/1 0/2 0/1 0/? 0/? 0/? 0/? )================ok ===( 17;0 1/1 0/2 0/1 0/? 0/? 0/? 0/? 0/? )================ok ===( 18;0 1/2 0/1 0/? 0/? 0/? 0/? 0/? 0/? )================Dubious\, test returned 1 (wstat 256\, 0x100) Failed 1/2 subtests ===( 20;0 1/1 0/? 0/? 0/? 0/? 0/? 0/? 0/? )================ok ===( 21;0 1/2 0/? 0/? 0/? 0/? 0/? 0/? 0/? )================ok ===( 22;0 0/3 0/2 0/? 0/? 0/? 0/? 0/? )=====================Directory 'test.lib' exists (it shouldn't yet) - removing it at t/pod2html-lib.pl line 14. ===( 28;0 2/3 1/2 1/2 1/1 1/1 0/1 0/1 0/? )================ok ===( 29;0 3/3 1/2 1/2 1/1 0/1 0/1 0/? )=====================ok ===( 30;0 3/3 2/2 1/2 0/1 0/1 0/? 0/? )=====================ok ===( 30;0 3/3 1/2 0/1 0/1 0/? 0/? )==========================ok ===( 30;0 1/2 0/1 0/1 0/? 0/? )===============================ok ===( 32;0 1/1 0/1 0/? 0/? )====================================ok ===( 33;0 1/1 0/? 0/? )=========================================ok ===( 34;0 1/1 0/? )==============================================ok ../ext/Pod-Html/t/podnoerr.t ... ok 1 - pod error section ok

Test Summary Report

---

../ext/Pod-Html/t/cache.t (Wstat​: 512 Tests​: 10 Failed​: 2)   Failed tests​: 4-5   Non-zero exit status​: 2 ../ext/Pod-Html/t/feature2.t (Wstat​: 256 Tests​: 2 Failed​: 1)   Failed test​: 2   Non-zero exit status​: 1 Files=18\, Tests=35\, 1 wallclock secs ( 0.05 usr 0.01 sys + 1.45 cusr 0.18 csys = 1.69 CPU) Result​: FAIL /home/jkeenan/perl

[jkeenan]

At least some of the problems reported in this ticket, which I originally filed in 2012, are still present today. Example:

$ cd t; TEST_JOBS=4 ./perl harness ../ext/Pod-Html/t/*.t; cd -
../ext/Pod-Html/t/anchorify.t .. ok                                     
===(       1;0  0/?  0/?  0/? )=========================================Directory 'test.lib' exists (it shouldn't yet) - removing it at ./t/pod2html-lib.pl line 14.
../ext/Pod-Html/t/cache.t ...... ok                                     
../ext/Pod-Html/t/crossref3.t .. ok                                     
../ext/Pod-Html/t/crossref2.t .. ok                                     
../ext/Pod-Html/t/crossref.t ... ok                                     
===(      15;0  1/3  0/2  0/?  0/? )====================================
#   Failed test 'misc pod-html --verbose warnings'
#   at t/feature2.t line 26.
#                   'scanning for directory cache
# scanning for directory cache
# caching directories for later use
# Converting input file /home/jkeenan/gitwork/perl/ext/Pod-Html/t/feature2.pod
# Cannot find file "crossref.*" directly under podpath, cannot find suitable replacement: link remains unresolved.
# '
#     doesn't match '(?^x:
#     \Acaching\ directories\ for\ later\ use\n
#     Converting\ input\ file\ \S+[/\\\]]feature2\.pod\n
#     Cannot\ find\ file\ "crossref\.\*"\ directly\ under\ podpath,\ cannot\ find
#     \ suitable\ replacement:\ link\ remains\ unresolved\.\n\z
#     )'
# Looks like you failed 1 test of 2.                                    
../ext/Pod-Html/t/eol.t ........ ok
../ext/Pod-Html/t/feature2.t ... Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/2 subtests 
../ext/Pod-Html/t/feature.t .... ok                                     
../ext/Pod-Html/t/htmldir1.t ... ok                                     
../ext/Pod-Html/t/htmldir2.t ... ok                                     
../ext/Pod-Html/t/htmldir3.t ... ok                                     
../ext/Pod-Html/t/htmldir4.t ... ok                                     
../ext/Pod-Html/t/htmldir5.t ... ok                                     
../ext/Pod-Html/t/htmlescp.t ... ok                                     
../ext/Pod-Html/t/htmllink.t ... ok                                     
../ext/Pod-Html/t/htmlview.t ... ok                                     
../ext/Pod-Html/t/poderr.t ..... ok                                     
../ext/Pod-Html/t/podnoerr.t ... ok   

Test Summary Report
-------------------
../ext/Pod-Html/t/feature2.t (Wstat: 256 Tests: 2 Failed: 1)
  Failed test:  2
  Non-zero exit status: 1
Files=18, Tests=35,  1 wallclock secs ( 0.04 usr  0.03 sys +  2.67 cusr  0.29 csys =  3.03 CPU)
Result: FAIL
/home/jkeenan/gitwork/perl

However, I no longer have the branch that I was working on when I filed the ticket. The laptop I was using has, for all practical purposes, been wiped. The "dromedary" development server I was using crashed and the code on it was unsalvageable.

I suspect the root problem here is that we're not using tempdirs and tempfiles to create the test .html files created during testing. Once some other Pod-Html tickets are resolved, I hope to explore that direction.

Thank you very much. Jim Keenan

[jkeenan]

From @jkeenan

This is a bug report for perl from jkeenan@​cpan.org, generated with the help of perlbug 1.39 running under perl 5.16.0. [snip]

This suggests to me that the problem is in the way t/htmldir3.t is formulated. My hunch is that the first test in t/htmldir3.t changes the environment in some way, not yet understood, that enables the second test to pass. But when we make that test the first one in a file, it fails.

This problem should be resolved. In the recent series of pull requests refactoring ext/Pod-Html, I converted all the tests to create tempfiles for their execution. I also added an expect_fail KVP to the testing infrastructure, so that you can specify that you expect a given POD-to-HTML conversion not to "succeed" (that is, match the expected output exactly).

So ext/Pod-Html/t/htmldir3.t now does the right thing. I will close this ticket within 3 days unless there is an objection.

Thank you very much. Jim Keenan