search

Perl / perl5

šŸŖ The Perl programming language
https://dev.perl.org/perl5/
Other
1.99k stars 559 forks source link

copy encoding settings when duping file descriptors (?) #14229

Open p5pRT opened 10 years ago

p5pRT commented 10 years ago

Migrated from rt.perl.org#123173 (status was 'open')

Searchable as RT123173$

p5pRT commented 10 years ago

From @jimav

This is a bug report for perl from jim.avera@​gmail.com\, generated with the help of perlbug 1.40 running under perl 5.20.1.

Should an open which dups another file descirptor copy it's encoding settings? I suspect that would "do the right thing" more often than not.

Here's a true story​: A library module has to run an external command which is noisy on stderr. To suppress the noise\, it does​:

  use File​::Spec​::Functions qw(devnull)

  open(my $oldSTDERR\, ">&STDERR");   open(STDERR\, ">"\, devnull());

  system("prog"\,"arg1"\,"arg2"\,...);

  open(STDERR\, ">&"\, $oldSTDERR)   close $oldSTDERR;

...almost exactly as shown in example code in the documentation of 'open'.

But doing that causes the "restored" STDERR to have default encodings\, not the encodings which the user may have instantiated\, perhaps via

  use open OUT => 'utf8'\, '​:std';

in the main module.

Yes\, the library module can do fork\, redirect STDERR in the child\, exec\, and wait\, and thereby avoid this problem. But it's quite a bit more complicated\, and not as portable.

Flags​:   category=core   severity=low

Site configuration information for perl 5.20.1​:

Configured by Debian Project at Fri Oct 10 14​:16​:26 UTC 2014.

Summary of my perl5 (revision 5 version 20 subversion 1) configuration​:  
  Platform​:   osname=linux\, osvers=3.2.0-37-generic\, archname=x86_64-linux-gnu-thread-multi   uname='linux batsu 3.2.0-37-generic #58-ubuntu smp thu jan 24 15​:28​:10 utc 2013 x86_64 x86_64 x86_64 gnulinux '   config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN -D_FORTIFY_SOURCE=2 -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Dldflags= -Wl\,-Bsymbolic-functions -Wl\,-z\,relro -Dlddlflags=-shared -Wl\,-Bsymbolic-functions -Wl\,-z\,relro -Dcccdlflags=-fPIC -Darchname=x86_64-linux-gnu -Dprefix=/usr -Dprivlib=/usr/share/perl/5.20 -Darchlib=/usr/lib/x86_64-linux-gnu/perl/5.20 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/x86_64-linux-gnu/perl5/5.20 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.20.1 -Dsitearch=/usr/local/lib/x86_64-linux-gnu/perl/5.20.1 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Duse64bitint -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Ud_ualarm -Uusesfio -Uusenm -Ui_libutil -Uversiononly -DDEBUGGING=-g -Doptimize=-O2 -Duseshrplib -Dlibperl=libperl.so.5.20.1 -des'   hint=recommended\, useposix=true\, d_sigaction=define   useithreads=define\, usemultiplicity=define   use64bitint=define\, use64bitall=define\, uselongdouble=undef   usemymalloc=n\, bincompat5005=undef   Compiler​:   cc='cc'\, ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'\,   optimize='-O2 -g'\,   cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv -fno-strict-aliasing -pipe -I/usr/local/include'   ccversion=''\, gccversion='4.9.1'\, gccosandvers=''   intsize=4\, longsize=8\, ptrsize=8\, doublesize=8\, byteorder=12345678   d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=16   ivtype='long'\, ivsize=8\, nvtype='double'\, nvsize=8\, Off_t='off_t'\, lseeksize=8   alignbytes=8\, prototype=define   Linker and Libraries​:   ld='cc'\, ldflags =' -fstack-protector -L/usr/local/lib'   libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.9/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib   libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt   perllibs=-ldl -lm -lpthread -lc -lcrypt   libc=libc-2.19.so\, so=so\, useshrplib=true\, libperl=libperl.so.5.20   gnulibc_version='2.19'   Dynamic Linking​:   dlsrc=dl_dlopen.xs\, dlext=so\, d_dlsymun=undef\, ccdlflags='-Wl\,-E'   cccdlflags='-fPIC'\, lddlflags='-shared -L/usr/local/lib -fstack-protector'

Locally applied patches​:   DEBPKG​:debian/cpan_definstalldirs - Provide a sensible INSTALLDIRS default for modules installed from CPAN.   DEBPKG​:debian/db_file_ver - http​://bugs.debian.org/340047 Remove overly restrictive DB_File version check.   DEBPKG​:debian/doc_info - Replace generic man(1) instructions with Debian-specific information.   DEBPKG​:debian/enc2xs_inc - http​://bugs.debian.org/290336 Tweak enc2xs to follow symlinks and ignore missing @​INC directories.   DEBPKG​:debian/errno_ver - http​://bugs.debian.org/343351 Remove Errno version check due to upgrade problems with long-running processes.   DEBPKG​:debian/libperl_embed_doc - http​://bugs.debian.org/186778 Note that libperl-dev package is required for embedded linking   DEBPKG​:fixes/respect_umask - Respect umask during installation   DEBPKG​:debian/writable_site_dirs - Set umask approproately for site install directories   DEBPKG​:debian/extutils_set_libperl_path - EU​:MM​: set location of libperl.a under /usr/lib   DEBPKG​:debian/no_packlist_perllocal - Don't install .packlist or perllocal.pod for perl or vendor   DEBPKG​:debian/prefix_changes - Fiddle with *PREFIX and variables written to the makefile   DEBPKG​:debian/fakeroot - Postpone LD_LIBRARY_PATH evaluation to the binary targets.   DEBPKG​:debian/instmodsh_doc - Debian policy doesn't install .packlist files for core or vendor.   DEBPKG​:debian/ld_run_path - Remove standard libs from LD_RUN_PATH as per Debian policy.   DEBPKG​:debian/libnet_config_path - Set location of libnet.cfg to /etc/perl/Net as /usr may not be writable.   DEBPKG​:debian/mod_paths - Tweak @​INC ordering for Debian   DEBPKG​:debian/module_build_man_extensions - http​://bugs.debian.org/479460 Adjust Module​::Build manual page extensions for the Debian Perl policy   DEBPKG​:debian/prune_libs - http​://bugs.debian.org/128355 Prune the list of libraries wanted to what we actually need.   DEBPKG​:fixes/net_smtp_docs - [rt.cpan.org #36038] http​://bugs.debian.org/100195 Document the Net​::SMTP 'Port' option   DEBPKG​:debian/perlivp - http​://bugs.debian.org/510895 Make perlivp skip include directories in /usr/local   DEBPKG​:debian/deprecate-with-apt - http​://bugs.debian.org/747628 Point users to Debian packages of deprecated core modules   DEBPKG​:debian/squelch-locale-warnings - http​://bugs.debian.org/508764 Squelch locale warnings in Debian package maintainer scripts   DEBPKG​:debian/skip-upstream-git-tests - Skip tests specific to the upstream Git repository   DEBPKG​:debian/patchlevel - http​://bugs.debian.org/567489 List packaged patches for 5.20.1-1 in patchlevel.h   DEBPKG​:debian/skip-kfreebsd-crash - http​://bugs.debian.org/628493 [perl #96272] Skip a crashing test case in t/op/threads.t on GNU/kFreeBSD   DEBPKG​:fixes/document_makemaker_ccflags - http​://bugs.debian.org/628522 [rt.cpan.org #68613] Document that CCFLAGS should include $Config{ccflags}   DEBPKG​:debian/find_html2text - http​://bugs.debian.org/640479 Configure CPAN​::Distribution with correct name of html2text   DEBPKG​:debian/perl5db-x-terminal-emulator.patch - http​://bugs.debian.org/668490 Invoke x-terminal-emulator rather than xterm in perl5db.pl   DEBPKG​:debian/cpan-missing-site-dirs - http​://bugs.debian.org/688842 Fix CPAN​::FirstTime defaults with nonexisting site dirs if a parent is writable   DEBPKG​:fixes/memoize_storable_nstore - [rt.cpan.org #77790] http​://bugs.debian.org/587650 Memoize​::Storable​: respect 'nstore' option not respected   DEBPKG​:debian/regen-skip - Skip a regeneration check in unrelated git repositories   DEBPKG​:fixes/regcomp-mips-optim - [perl #122817] http​://bugs.debian.org/754054 Downgrade the optimization of regcomp.c on mips and mipsel due to a gcc-4.9 bug   DEBPKG​:debian/makemaker-pasthru - http​://bugs.debian.org/758471 Pass LD settings through to subdirectories   DEBPKG​:fixes/perldoc-less-R - [rt.cpan.org #98636] http​://bugs.debian.org/758689 Tell the 'less' pager to allow terminal escape sequences   DEBPKG​:fixes/hurd_test_todo_socket.t - [perl #122657] http​://bugs.debian.org/758718 Disable failing GNU/Hurd test in t/io/socket.t   DEBPKG​:fixes/pod_man_reproducible_date - http​://bugs.debian.org/759405 Support POD_MAN_DATE in Pod​::Man for the left-hand footer   DEBPKG​:fixes/data_dump_infinite_recurse - [19be3be] don't recurse infinitely in Data​::Dumper

@​INC for perl 5.20.1​:   /home/jima/lib/perl   /home/jima/perl5/lib/perl5/x86_64-linux-gnu-thread-multi   /home/jima/perl5/lib/perl5/x86_64-linux-gnu-thread-multi   /home/jima/perl5/lib/perl5   /etc/perl   /usr/local/lib/x86_64-linux-gnu/perl/5.20.1   /usr/local/share/perl/5.20.1   /usr/lib/x86_64-linux-gnu/perl5/5.20   /usr/share/perl5   /usr/lib/x86_64-linux-gnu/perl/5.20   /usr/share/perl/5.20   /usr/local/lib/site_perl   .

Environment for perl 5.20.1​:   HOME=/home/jima   LANG=en_US.UTF-8   LANGUAGE=en_US   LD_LIBRARY_PATH=/home/jima/local/lib   LOGDIR (unset)   PATH=/home/jima/perl5/bin​:/home/jima/bin​:/home/jima/local/bin​:/home/jima/jima_tools/x86_64/bin​:/home/jima/jima_tools/bin​:/opt/Adobe/Reader9/bin​:/usr/bin​:/bin​:/usr/sbin​:/sbin​:/usr/bin/X11​:/usr/local/bin​:/usr/local/sbin​:/usr/games​:/usr/local/games​:/usr/lib/jvm/java-7-oracle/bin​:/usr/lib/jvm/java-7-oracle/db/bin​:/usr/lib/jvm/java-7-oracle/jre/bin​:.   PERL5LIB=/home/jima/lib/perl​:/home/jima/perl5/lib/perl5/x86_64-linux-gnu-thread-multi​:/home/jima/perl5/lib/perl5   PERL_BADLANG (unset)   PERL_LOCAL_LIB_ROOT=/home/jima/perl5   PERL_MB_OPT=--install_base /home/jima/perl5   PERL_MM_OPT=INSTALL_BASE=/home/jima/perl5   SHELL=/bin/bash

p5pRT commented 10 years ago

From @ap

* via RT \perlbug\-followup@​perl\.org [2014-11-11 19​:50]​:

Should an open which dups another file descirptor copy it's encoding settings?

Sometimes yes and sometimes no.

I suspect that would "do the right thing" more often than not.

More often\, but not always. So just doing the opposite from what it does now will be wrong too.

There must be a way to explicitly say that you do want the PerlIO layers cloned\, and another that donā€™t want them cloned.

Which of these should be the default? Good question.

In the short term\, there at least needs to be an easy way to tell PerlIO to make the layers on one filehandle be exactly like the layers on some other handle. (Leon?)

Here's a true story​:

I suspect that the &GLOBNAME form indicates a dup that will likely need the layers cloned and the &=X form indicates a dup that will likely need the cloned handle raw\, or at least is much more likely to. Or maybe the divide is not &GLOBNAME vs &=X but just &STDOUT and &STDERR vs all other dups. I donā€™t do this nor see this often enough to have a sense of this.

If dup gets a feature where to explicitly not-/clone layers\, it may be good to make default depend on the form used\, for good DWIM and Huffman. Or it may be too surprising. (If it is even feasible backcompat-wise.)

Regards\, -- Aristotle Pagaltzis // \<http​://plasmasturm.org/>

p5pRT commented 10 years ago

The RT System itself - Status changed from 'new' to 'open'