Filehandle capture confuses PerlIO layers

[p5pRT]

Migrated from rt.perl.org#114404 (status was 'open')

Searchable as RT114404$

[p5pRT]

From @xdg

Created by @xdg

When Capture​::Tiny saves\, redirects and restores filehandles to capture output\, it also duplicates the original filehandle layers on the capturing filehandle. Something in this process appears to confuse PerlIO. In the example file below\, PerlIO​::get_layers shows the same layers before and after\, yet afterwards a wide character in print warning is issued. I have confirmed this issue in Perl 5.14.2\, Perl 5.10.1 and Perl 5.8.9. (Originally reported against Perl 5.16.0 in Capture​::Tiny bug rt.cpan.org #78819)

  use Encode;   use Capture​::Tiny qw(capture);

  sub say { print @​_\, "\n" }

  my $foo = Encode​::decode_utf8("\xC5\x82");

  binmode( STDOUT\, "​:utf8" );

  out('PRE');   capture { say "hello world" };   out('POST');

  sub out {   say STDOUT "$_[0]​:";   say STDOUT "fd " . fileno(STDOUT) . " has " . join '\, '\, PerlIO​::get_layers( \*STDOUT );   say STDOUT $foo;   say STDOUT '';   }

  ##### OUTPUT #####   #PRE​:   #fd 1 has unix\, perlio\, utf8   #ł   #   #POST​:   #fd 1 has unix\, perlio\, utf8   #Wide character in print at /home/david/perl5/perlbrew/perls/perl-5.14.2/lib/5.14.2/x86_64-linux/IO/Handle.pm line 430.   #ł

Perl Info

``` Flags: category=core severity=medium Site configuration information for perl 5.14.2: Configured by david at Thu Jan 5 16:18:53 EST 2012. Summary of my perl5 (revision 5 version 14 subversion 2) configuration: Platform: osname=linux, osvers=3.0.0-14-generic, archname=x86_64-linux uname='linux vulcan 3.0.0-14-generic #23-ubuntu smp mon nov 21 20:28:43 utc 2011 x86_64 x86_64 x86_64 gnulinux ' config_args='-de -Dprefix=/home/david/perl5/perlbrew/perls/perl-5.14.2' hint=recommended, useposix=true, d_sigaction=define useithreads=undef, usemultiplicity=undef useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef use64bitint=define, use64bitall=define, uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cc', ccflags ='-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-O2', cppflags='-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include' ccversion='', gccversion='4.6.1', gccosandvers='' intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16 ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='cc', ldflags =' -fstack-protector -L/usr/local/lib' libpth=/usr/local/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /usr/lib libs=-lnsl -ldb -ldl -lm -lcrypt -lutil -lc perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc libc=, so=so, useshrplib=false, libperl=libperl.a gnulibc_version='2.13' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E' cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector' Locally applied patches: @INC for perl 5.14.2: /home/david/perl5/perlbrew/perls/perl-5.14.2/lib/site_perl/5.14.2/x86_64-linux /home/david/perl5/perlbrew/perls/perl-5.14.2/lib/site_perl/5.14.2 /home/david/perl5/perlbrew/perls/perl-5.14.2/lib/5.14.2/x86_64-linux /home/david/perl5/perlbrew/perls/perl-5.14.2/lib/5.14.2 . Environment for perl 5.14.2: HOME=/home/david LANG=en_US.UTF-8 LANGUAGE=en_US:en LC_COLLATE=C LC_CTYPE=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/home/david/perl5/perlbrew/bin:/home/david/perl5/perlbrew/perls/perl-5.14.2/bin:~/bin:~/git/utility-scripts:/opt/perl/current/bin:/home/david/bin:/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/opt/vagrant/bin:. PERLBREW_BASHRC_VERSION=0.39 PERLBREW_HOME=/home/david/.perlbrew PERLBREW_MANPATH=/home/david/perl5/perlbrew/perls/perl-5.14.2/man PERLBREW_PATH=/home/david/perl5/perlbrew/bin:/home/david/perl5/perlbrew/perls/perl-5.14.2/bin PERLBREW_PERL=perl-5.14.2 PERLBREW_ROOT=/home/david/perl5/perlbrew PERLBREW_VERSION=0.46 PERL_BADLANG (unset) PERL_EXTUTILS_AUTOINSTALL=--defaultdeps SHELL=/bin/bash ```

[p5pRT]

From @xdg

Note -- triggering this behavior requires Capture​::Tiny 0.18 or earlier\, as later versions force a relayering back on top of restored filehandles as a workaround to this bug.

Just because Capture​::Tiny has a workaround does not resolve this issue. There is still no reason why PerlIO​::get_layers should be reporting layers that aren't actually in effect on a handle.

-- David

[p5pRT]

@xdg - Status changed from 'new' to 'open'

[p5pRT]

From @dmcbride

On Monday August 6 2012 5​:18​:48 PM David Golden wrote​:

When Capture​::Tiny saves\, redirects and restores filehandles to capture output\, it also duplicates the original filehandle layers on the capturing filehandle. Something in this process appears to confuse PerlIO. In the example file below\, PerlIO​::get_layers shows the same layers before and after\, yet afterwards a wide character in print warning is issued. I have confirmed this issue in Perl 5.14.2\, Perl 5.10.1 and Perl 5.8.9. (Originally reported against Perl 5.16.0 in Capture​::Tiny bug rt.cpan.org #78819)

This is a duplicate of https://rt-archive.perl.org/perl5/Ticket/Display.html?id=113982

use Encode; use Capture​::Tiny qw(capture);

sub say { print @​_\, "\n" }

my $foo = Encode​::decode_utf8("\xC5\x82");

binmode( STDOUT\, "​:utf8" );

out('PRE'); capture { say "hello world" }; out('POST');

sub out { say STDOUT "$_[0]​:"; say STDOUT "fd " . fileno(STDOUT) . " has " . join '\, '\, PerlIO​::get_layers( \*STDOUT ); say STDOUT $foo;

You need to add "\, output => 1" here​:

  say "fd " . fileno(STDOUT) . " has " . join '\, '\, PerlIO​::get_layers( \*STDOUT\, output => 1 );

Then you see the difference. Yes\, I faced the same quandary with the previous bug report.

(This may also mean that the fix/workaround to Capture​::Tiny might be off in that it's only checking the input layers\, not the output layers\, though I'm assured that most of the time they're the same.)

say STDOUT '';

}

##### OUTPUT ##### #PRE​: #fd 1 has unix\, perlio\, utf8 #ł # #POST​: #fd 1 has unix\, perlio\, utf8 #Wide character in print at

[p5pRT]

From @xdg

On Mon\, Aug 6\, 2012 at 11​:11 PM\, Darin McBride \dmcbride@​cpan\.org wrote​:

This is a duplicate of https://rt-archive.perl.org/perl5/Ticket/Display.html?id=113982

I don't think it's quite the same\, as this bug also points out that get_layers is reporting it incorrectly.

I've marked them as referring to each other\, so whoever fixes it can close them both.

-- David

[p5pRT]

From @Leont

On Tue\, Aug 7\, 2012 at 6​:11 AM\, Darin McBride \dmcbride@​cpan\.org wrote​:

(This may also mean that the fix/workaround to Capture​::Tiny might be off in that it's only checking the input layers\, not the output layers\, though I'm assured that most of the time they're the same.)

The layers shouldn't be different. binmode explicitly tries to make sure of that if the handles are different. The reason why have this duality is that on for example sockets input and output need separate buffers.

Leon

[p5pRT]

From @Leont

On Tue\, Aug 7\, 2012 at 6​:15 AM\, David Golden \xdaveg@​gmail\.com wrote​:

I don't think it's quite the same\, as this bug also points out that get_layers is reporting it incorrectly.

I don't see how it's incorrect\, it's just very confusing.

Leon