pod2man generates invalid roff

p5pRT commented 12 years ago

Migrated from rt.perl.org#111448 (status was 'open')

Searchable as RT111448$

p5pRT commented 12 years ago

From tchrist@perl.com

On the systems I tested it on\, the manpages whose pods contain non-ASCII are botched\, some to the point of complete illegibility.

This is wrong:

$ man perlhist| grep var Steve Hay\, Matt S Trout\, David Golden\, Florian Ragwitz\, Tatsuhiko Miyagawa\, Chris "BinGOs" Williams\, Zefram and var Arnfjoer` Bjarmason. Chip Salzenberg gets credit for that\, with a nod to his cow orker\, David Croy. We had passed around various names (baton\, token\, hot potato) but none caught on. var 5.13.10 2011-Feb-20

And this is intolerable:

$ man perlcn PERLCN(1) Perl Programmers Reference Guide PERLCN(1)

NAME perlcn - XXXX Perl XX

DESCRIPTION XXXX Perl XXX!

X 5.8.0 XXX\, Perl XXXXXX Unicode (XXX) XX\, XXXXXXXXXXXXXXXXXXX; CJK (XXX) XXXXXXXX. Unicode XXXXXXX\, XXXXXXXXXXXX: XXXX\, XXXX\, XXXXXXXX (XXX\, XXXX\, XXXX\, XXXX\, XXX\, XXXX\, XX). XXXXXXXXXXXXXX (X PC XXXX).

Perl XXX Unicode XXXX. XXX Perl XXXXXXXXXX Unicode XX; Perl XXXXXX (XXXXXXXXX) XXX Unicode XXXX. XXXXXXX\, XXXXX Unicode XXXXXXXXXXXX\, Perl XXX Encode XXXX\, XXXXXXXXXXXXXXXXXXX.

The bug is that while you are allowed to and encouraged to declare your podfiles' encoding\, this is severely disrespected.

The following podpages show the problem:

perlebcdic.pod perlgit.pod perlhist.pod perlpodspec.pod perlthrtut.pod

perl588delta.pod perl5100delta.pod perl5120delta.pod perl5121delta.pod perl5122delta.pod perl5123delta.pod perl5124delta.pod perl5140delta.pod perl5141delta.pod perl5142delta.pod perl5150delta.pod perl5151delta.pod perl5152delta.pod perl5153delta.pod perl5154delta.pod perl5156delta.pod perl5157delta.pod perl5158delta.pod

perlcn.pod perljp.pod perlko.pod perltw.pod

--tom

Summary of my perl5 (revision 5 version 14 subversion 0) configuration:
Platform: osname=openbsd\, osvers=4.4\, archname=OpenBSD.i386-openbsd uname='openbsd chthon 4.4 generic#0 i386 ' config_args='-des' hint=recommended\, useposix=true\, d_sigaction=define useithreads=undef\, usemultiplicity=undef useperlio=define\, d_sfio=undef\, uselargefiles=define\, usesocks=undef use64bitint=undef\, use64bitall=undef\, uselongdouble=undef usemymalloc=y\, bincompat5005=undef Compiler: cc='cc'\, ccflags ='-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'\, optimize='-O2'\, cppflags='-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include' ccversion=''\, gccversion='3.3.5 (propolice)'\, gccosandvers='openbsd4.4' intsize=4\, longsize=4\, ptrsize=4\, doublesize=8\, byteorder=1234 d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=12 ivtype='long'\, ivsize=4\, nvtype='double'\, nvsize=8\, Off_t='off_t'\, lseeksize=8 alignbytes=4\, prototype=define Linker and Libraries: ld='cc'\, ldflags ='-Wl\,-E -fstack-protector -L/usr/local/lib' libpth=/usr/local/lib /usr/lib libs=-lgdbm -lm -lutil -lc perllibs=-lm -lutil -lc libc=/usr/lib/libc.so.48.0\, so=so\, useshrplib=false\, libperl=libperl.a gnulibc_version='' Dynamic Linking: dlsrc=dl_dlopen.xs\, dlext=so\, d_dlsymun=undef\, ccdlflags=' ' cccdlflags='-DPIC -fPIC '\, lddlflags='-shared -fPIC -L/usr/local/lib -fstack-protector'

Characteristics of this binary (from libperl): Compile-time options: MYMALLOC PERL_DONT_CREATE_GVSV PERL_MALLOC_WRAP PERL_PRESERVE_IVUV USE_LARGE_FILES USE_PERLIO USE_PERL_ATOF Built under openbsd Compiled at Jun 11 2011 11:48:28 %ENV: PERL_UNICODE="SA" @INC: /usr/local/lib/perl5/site_perl/5.14.0/OpenBSD.i386-openbsd /usr/local/lib/perl5/site_perl/5.14.0 /usr/local/lib/perl5/5.14.0/OpenBSD.i386-openbsd /usr/local/lib/perl5/5.14.0 /usr/local/lib/perl5/site_perl/5.12.3 /usr/local/lib/perl5/site_perl/5.11.3 /usr/local/lib/perl5/site_perl/5.10.1 /usr/local/lib/perl5/site_perl/5.10.0 /usr/local/lib/perl5/site_perl/5.8.7 /usr/local/lib/perl5/site_perl/5.8.0 /usr/local/lib/perl5/site_perl/5.6.0 /usr/local/lib/perl5/site_perl/5.005 /usr/local/lib/perl5/site_perl .

p5pRT commented 11 years ago

From @jkeenan

On Wed Feb 29 13:04:43 2012\, tom christiansen wrote:

On the systems I tested it on\, the manpages whose pods contain non- ASCII are botched\, some to the point of complete illegibility.

This is wrong:
$ man perlhist| grep var
   Steve Hay\, Matt S Trout\, David Golden\, Florian Ragwitz\,
Tatsuhiko Miyagawa\, Chris "BinGOs" Williams\, Zefram and var Arnfjoer` Bjarmason. Chip Salzenberg gets credit for that\, with a nod to his cow orker\, David Croy. We had passed around various names (baton\, token\, hot potato) but none caught on. var 5.13.10 2011-Feb-20

And this is intolerable:
$ man perlcn
PERLCN$1$                                            Perl
Programmers Reference Guide PERLCN(1)
NAME
   perlcn \- XXXX Perl XX

DESCRIPTION
   XXXX Perl XXX\!

   X 5\.8\.0 XXX\, Perl XXXXXX Unicode $XXX$ XX\, XXXXXXXXXXXXXXXXXXX;
CJK (XXX) XXXXXXXX. Unicode XXXXXXX\, XXXXXXXXXXXX: XXXX\, XXXX\, XXXXXXXX (XXX\, XXXX\, XXXX\, XXXX\, XXX\, XXXX\, XX). XXXXXXXXXXXXXX (X PC XXXX).
   Perl XXX Unicode XXXX\. XXX Perl XXXXXXXXXX Unicode XX; Perl XXXXXX
(XXXXXXXXX) XXX Unicode XXXX. XXXXXXX\, XXXXX Unicode XXXXXXXXXXXX\, Perl XXX Encode XXXX\, XXXXXXXXXXXXXXXXXXX.

The bug is that while you are allowed to and encouraged to declare your podfiles' encoding\, this is severely disrespected.

The following podpages show the problem:
perlebcdic\.pod
perlgit\.pod
perlhist\.pod
perlpodspec\.pod
perlthrtut\.pod

perl588delta\.pod
perl5100delta\.pod
perl5120delta\.pod
perl5121delta\.pod
perl5122delta\.pod
perl5123delta\.pod
perl5124delta\.pod
perl5140delta\.pod
perl5141delta\.pod
perl5142delta\.pod
perl5150delta\.pod
perl5151delta\.pod
perl5152delta\.pod
perl5153delta\.pod
perl5154delta\.pod
perl5156delta\.pod
perl5157delta\.pod
perl5158delta\.pod

perlcn\.pod
perljp\.pod
perlko\.pod
perltw\.pod
--tom

Is this a problem that can be fixed in the 'pod2man' utility?

Or does it require corrections of individual .pod files?

Or both?

Or neither?

Thank you very much. Jim Keenan

p5pRT commented 11 years ago

The RT System itself - Status changed from 'new' to 'open'

p5pRT commented 11 years ago

From @mrallen1

On Wed Sep 26 19:07:49 2012\, jkeenan wrote:

On Wed Feb 29 13:04:43 2012\, tom christiansen wrote:

On the systems I tested it on\, the manpages whose pods contain non- ASCII are botched\, some to the point of complete illegibility.

Is this a problem that can be fixed in the 'pod2man' utility?

Or does it require corrections of individual .pod files?

Or both?

Or neither?

Hi. I'm the current maintainer of Pod::Perldoc. In 90% of the cases where the output from pod2man is displayed as "X" instead of an proper glyph\, it is typically groff itself which botches the handling non-ASCII characters.

Getting clean non-ASCII characters typically requires a groff >= 1.20.1 and within the context of perldoc (at least) a recent version (>=3.18). As far as pod2man is concerned\, I believe it handles non-ASCII characters as well as it can given the various points where encoding matters to display which are (among others)\, the shell itself\, perl\, groff\, the pager program\, and whatever terminal one is using.

So I would say this report is inconclusive. It may well be that pod2man is generating proper roff output\, and groff is mangling it when it tries to format and display it.

Mark

p5pRT commented 11 years ago

From @mrallen1

On Wed Sep 26 19:07:49 2012\, jkeenan wrote:

On Wed Feb 29 13:04:43 2012\, tom christiansen wrote:

On the systems I tested it on\, the manpages whose pods contain non- ASCII are botched\, some to the point of complete illegibility.

Is this a problem that can be fixed in the 'pod2man' utility?

Or does it require corrections of individual .pod files?

Or both?

Or neither?

Hi. I'm the current maintainer of Pod::Perldoc. In 90% of the cases where the output from pod2man is displayed as "X" instead of an proper glyph\, it is typically groff itself which botches the handling non-ASCII characters.

Getting clean non-ASCII characters typically requires a groff >= 1.20.1 and within the context of perldoc (at least) a recent version (>=3.18). As far as pod2man is concerned\, I believe it handles non-ASCII characters as well as it can given the various points where encoding matters to display which are (among others)\, the shell itself\, perl\, groff\, the pager program\, and whatever terminal one is using.

So I would say this report is inconclusive. It may well be that pod2man is generating proper roff output\, and groff is mangling it when it tries to format and display it.

Mark

Perl / perl5