Open p5pRT opened 8 years ago
I saw the issue when I use File::Slurper + Data::Table together in one script. I then simplified the demo case to be like below. Basically it's like\, you open a file in with some encoding\, and set $PerlIO::encoding::fallback in that "local"\, then out of that "local" scope you open another file with some encoding\, you get an infinite loop when reading the second file.
use 5.022;
my $file1 = '/tmp/foo'; my $file2 = '/tmp/bar'; system("echo bar > $file2");
{ # what's in File::Slurper my $layer = ':raw:encoding(utf-8)'; local $PerlIO::encoding::fallback = 1; open my $fh1\, ">$layer"\, $file1 or die $!; close $fh1 or die $!; }
{ # what's in Data::Table open(my $fh2\, $file2) or die $!; binmode $fh2\, ":encoding(UTF-8)"; while (my $line = \<$fh2>) { warn $line; } }
It can also be simplified to like below.
use 5.022;
my $file1 = '/tmp/foo'; my $file2 = '/tmp/bar'; system("echo foo > $file1"); system("echo bar > $file2");
{ local $PerlIO::encoding::fallback = 1; open my $fh1\, "\<:encoding(utf-8)"\, $file1 or die $!; close $fh1 or die $!; }
{ open my $fh2\, '\<:encoding(utf-8)'\, $file2 or die $!; while (my $line = \<$fh2>) { warn $line; } }
On Mon Jan 04 06:12:19 2016\, zhouzhen1@gmail.com wrote:
I saw the issue when I use File::Slurper + Data::Table together in one script. I then simplified the demo case to be like below. Basically it's like\, you open a file in with some encoding\, and set $PerlIO::encoding::fallback in that "local"\, then out of that "local" scope you open another file with some encoding\, you get an infinite loop when reading the second file.
The problem is that since PerlIO::encoding is loaded within the local's block\, $PerlIO::encoding::fallback is restored to it's original value at entry to the block - undef.
This can be prevented by explicitly loading PerlIO::encoding before localizing $PerlIO::encoding::fallback
Also\, since PerlIO::encoding is loaded *after* you initialized fallback\, it's being reset to the default set in PerlIO/encoding.pm before the value is used.
That said\, PerlIO::encoding shouldn't go into an infinite loop\, the attached patch fixes that for this case by ensuring the STOP_AT_PARTIAL flag is set\, which is needed for decoding.
Tony
The RT System itself - Status changed from 'new' to 'open'
Migrated from rt.perl.org#127149 (status was 'open')
Searchable as RT127149$