p5pRT
commented
10 years ago From @bulk88
Created by @bulk88
Some time in the last couple days this test on blead started failing on me. WinXP\, 32 bit\, VC 2003.
------------------------------------------------------------------ Test Summary Report ------------------- run/locale.t (Wstat: 0 Tests: 16 Failed: 16) Failed tests: 1-16 Files=2402\, Tests=694552\, 2421 wallclock secs (77.34 usr + 4.25 sys = 81.59 CPU ) Result: FAIL NMAKE : fatal error U1077: '..\perl.exe' : return code '0x10' Stop.
C:\perl519\src\win32> ------------------------------------------------------------------
http://perl5.git.perl.org/perl.git/commit/d369fd5b026662cae636af9907037fdd15fc4adb maybe this is related?
----------------------------------------------------------------- C:\perl519\src\t>..\perl -I..\lib harness run/locale.t run/locale.t .. # Failed test 1 - no locales where LC_NUMERIC breaks at ./test.p l line 954 # got undef # expected "" # PROG: # for (qw(Afrikaans Albanian American Basque Bulgarian C Croatian Czech Danish D utch English Estonian Finnish Galician German Greek Hungarian Indonesian Italian Latvian Lithuanian Norwegian Polish Russian Serbian Slovak Slovenian Swedish Tu rkish Welsh af da en fi fy ga hu it la mo no no\@nynorsk ro ru sl)) { # use POSIX qw(locale_h); # use locale; # setlocale(LC_NUMERIC\, "$_") or next; # my $s = sprintf "%g %g"\, 3.1\, 3.1; # next if $s eq '3.1 3.1' || $s =~ /^(3.+1) \1$/; # print "$_ $s\n"; # } # STATUS: -1 # Failed test 2 - LC_NUMERIC without environment nor setlocale() has no effect i n any locale at ./test.pl line 954 # got undef run/locale.t .. 1/16 # expected "" # PROG: # for (qw(Afrikaans Albanian American Basque Bulgarian C Croatian Czech Danish D utch English Estonian Finnish Galician German Greek Hungarian Indonesian Italian Latvian Lithuanian Norwegian Polish Russian Serbian Slovak Slovenian Swedish Tu rkish Welsh af da en fi fy ga hu it la mo no no\@nynorsk ro ru sl)) { # use POSIX qw(locale_h); # use locale; # my $in = 4.2; # my $s = sprintf "%g"\, $in; # avoid any constant folding bugs # next if $s eq "4.2"; # print "$_ $s\n"; # } # STATUS: -1 # Failed test 3 - format() does not look at LC_NUMERIC without 'use locale' at . /test.pl line 954 # got undef # expected "4.2" # PROG: # format STDOUT = # @.# # 4.179 # . # write; # STATUS: -1 # Failed test 4 - format() looks at LC_NUMERIC with 'use locale' at ./test.pl li ne 954 # got undef # expected "4\,2" # PROG: # use locale; # format STDOUT = # @.# # 4.179 # . # write; # STATUS: -1 # Failed test 5 - format() looks at LC_NUMERIC with 'use locale ":not_characters "' at ./test.pl line 954 # got undef # expected "4\,2" # PROG: # use locale ":not_characters"; # format STDOUT = # @.# # 4.179 # . # write; # STATUS: -1 # Failed test 6 - too late to look at the locale at write() time at ./test.pl li ne 954 # got undef # expected "4.2" # PROG: # format STDOUT = # @.# # 4.179 # . # { require locale; import locale; write; } # STATUS: -1 # Failed test 7 - too late to ignore the locale at write() time at ./test.pl lin e 954 # got undef # expected "4\,2" # PROG: # use locale; # format STDOUT = # @.# # 4.179 # . # { no locale; write; } # STATUS: -1 # Failed test 8 - version does not clobber version at ./test.pl line 954 # got undef # expected "" # PROG: # BEGIN { # if("define") { # require locale; import locale; # } # } # use POSIX; # my $i = 0.123; # POSIX::setlocale(POSIX::LC_NUMERIC()\,"Albanian"); # $a = sprintf("%.2f"\, $i); # require version; # $b = sprintf("%.2f"\, $i); # print ".$a $b" unless $a eq $b # STATUS: -1 # Failed test 9 - version does not clobber version (via eval) at ./test.pl line 954 # got undef # expected "" # PROG: # use locale; # use POSIX; # my $i = 0.123; # POSIX::setlocale(POSIX::LC_NUMERIC()\,"Albanian"); # $a = sprintf("%.2f"\, $i); # eval "use v5.0.0"; # $b = sprintf("%.2f"\, $i); # print "$a $b" unless $a eq $b # STATUS: -1 # Failed test 10 - sprintf() and printf() look at LC_NUMERIC regardless of const ant folding at ./test.pl line 954 # got undef # expected "4\,2 4\,2 4\,2 4\,2 " # PROG: # use locale; # use POSIX qw(locale_h); # setlocale(LC_NUMERIC\, ""); # my $in = 4.2; # printf("%g %g %s %s "\, $in\, 4.2\, sprintf("%g"\, $in)\, sprintf("%g"\, 4 .2)); # STATUS: -1 # Failed test 11 - No compile error on v-strings when setting the locale to non- dot radix at compile time when default environment has non-dot radix at ./test.p l line 954 # got undef # expected "C" # PROG: # use POSIX qw(locale_h); # # BEGIN { setlocale(LC_NUMERIC\, "Albanian"); }; # setlocale(LC_ALL\, "C"); # use 5.008; # print setlocale(LC_NUMERIC); # STATUS: -1 # Failed test 12 - Passed in LC_NUMERIC is valid at compilation time at ./test.p l line 954 # got undef # expected "Albanian" # PROG: # use POSIX qw(locale_h); # # BEGIN { print setlocale(LC_NUMERIC)\, " # "; }; # STATUS: -1 # Failed test 13 - Radix print properly in locale scope\, and without at ./test.p l line 954 # got undef # expected "1\,5\n1.5" # PROG: # my $i = 1.5; # { # use locale; # use POSIX; # POSIX::setlocale(POSIX::LC_NUMERIC()\,"Albanian"); # print $i\, " # "; # } # print $i\, " # "; # STATUS: -1 # Failed test 14 - Can do math when radix is a comma at ./test.pl line 954 # got undef # expected "1\,5\n2\,5" # PROG: # my $i = 1.5; # Should be exactly representable as a base 2 # # fraction\, so can use 'eq' below # use locale; # use POSIX; # POSIX::setlocale(POSIX::LC_NUMERIC()\,"Albanian"); # print $i\, " # "; # $i += 1; # print $i\, " # "; # STATUS: -1 # Failed test 15 - POSIX::strtod() uses underlying locale at ./test.pl line 954 # got undef # expected "1.5" # PROG: # use POSIX; # POSIX::setlocale(POSIX::LC_NUMERIC()\,"Albanian"); # my $one_point_5 = POSIX::strtod("1\,5"); # $one_point_5 =~ s/0+$//; # Remove any trailing zeros # print $one_point_5\, " # "; # STATUS: -1 # Failed test 16 - /il matching of [bracketed] doesn't skip POSIX class if fails individ char at ./test.pl line 954 # got undef # expected "1" # PROG: # use locale; # use POSIX; # POSIX::setlocale(POSIX::LC_CTYPE()\,"C"); # print "h" =~ /[g\w]/i || 0; # print "\n"; # STATUS: -1 run/locale.t .. Failed 16/16 subtests
Test Summary Report ------------------- run/locale.t (Wstat: 0 Tests: 16 Failed: 16) Failed tests: 1-16 Files=1\, Tests=16\, 1 wallclock secs ( 0.03 usr + 0.00 sys = 0.03 CPU) Result: FAIL
C:\perl519\src\t> ----------------------------------------------------------------- C:\perl519\src\t>git log -n 4 WARNING: terminal is not fully functional commit c02d66974dbe736ac96e0ee464b2ee7948935ffe Author: Daniel Dragan \bulk88@​hotmail\.com Date: Fri Jan 10 16:32:37 2014 -0500
b88 bld
commit fb29cc72fe9ef39100123477fffae4fad34866b7 Author: Karl Williamson \public@​khwilliamson\.com Date: Mon Jan 27 22:42:25 2014 -0700
regcomp.c: Change a variable and flag bit names
The meaning of these was expanded two commits ago\, so update the name to reflect this\, to prevent future confusion
commit baa60164aa329dbdc2e418a7743d51a081db237e Author: Karl Williamson \public@​khwilliamson\.com Date: Mon Jan 27 22:30:29 2014 -0700
White-space\, comments only
This mostly indents and outdents base on blocks added or removed by the previous commit. But there are a few comment changes and vertical alignment of macro backslash continuation characters\, and other white-space changes : commit 31f05a37c4e9c37a7263491f2fc0237d836e1a80 Author: Karl Williamson \public@​khwilliamson\.com Date: Mon Jan 27 15:35:00 2014 -0700
Work properly under UTF-8 LC_CTYPE locales
This large (sorry\, I couldn't figure out how to meaningfully split it up) commit causes Perl to fully support LC_CTYPE operations (case changing\, character classification) in UTF-8 locales.
As a side effect it resolves [perl #56820].
The basics are easy\, but there were a lot of details\, and one troublesome edge case discussed below.
What essentially happens is that when the locale is changed to a UTF-8 one\, a global variable is set TRUE (FALSE when changed to a non-UTF-8 locale). Within the scope of 'use locale'\, this variable is checked\, and if TRUE\, the code that Perl uses for non-locale behavior is used instead of the code for locale behavior. Since Perl's internal representation is UTF-8\, we get UTF-8 behavior for a UTF-8 locale.
More work had to be done for regular expressions. There are three cases.
1) The character classes \w\, [[:punct:]] needed no extra work\, as the changes fall out from the base work.
2) Strings that are to be matched case-insensitively. These form EXACTFL regops (nodes). Notice that if such a string contains only characters above-Latin1 that match only themselves\, that the node can be downgraded to an EXACT-only node\, which presents better optimization possibilities\, as we now have a fixed string known at compile time to be required to be in the target string to match. Similarly if all characters in the string match only other above-Latin1 characters case-insensitively\, the node can be downgraded to a regular EXACTFU node (match\, folding\, using Unicode\, not locale\, rules). The code changes for this could be done without accepting UTF-8 locales fully\, but there were edge cases which needed to be handled differently if I stopped there\, so I continued on.
In an EXACTFL node\, all such characters are now folded at compile time (just as before this commit)\, while the other characters whose folds are locale-dependent are left unfolded. This means that they have to be folded at execution time based on the locale in effect at the moment. Again\, this isn't a change from before. The difference is that now some of the folds that need to be done at execution time (in regexec) are potentially multi-char. Some of the code in regexec was trivial to extend to account for this because of existing infrastructure\, but the part dealing with regex quantifiers\, had to have more work.
Also the code that joins EXACTish nodes together had to be expanded to account for the possibility of multi-character folds within locale handling. This was fairly easy\, because it already has infrastructure to handle these under somewhat different circumstances.
3) In bracketed character classes\, represented by ANYOF nodes\, a new inversion list was created giving the characters that should be matched by this node when the runtime locale is UTF-8. The list is ignored except under that circumstance. To do this\, I created a new ANYOF type which has an extra SV for the inversion list.
The edge case that caused the most difficulty is folding involving the MICRO SIGN\, U+00B5. It folds to the GREEK SMALL LETTER MU\, as does the GREEK CAPITAL LETTER MU. The MICRO SIGN is the only 0-255 range character that folds to outside that range. The issue is that it doesn't naturally fall out that it will match the CAP MU. If we let the CAP MU fold to the samll mu at compile time (which it can because both are above-Latin1 and so the fold is the same no matter what locale is in effect)\, it could appear that the regnode can be downgraded away from EXACTFL to EXACTFU\, but doing so would cause the MICRO SIGN to not case insensitvely match the CAP MU. This could be special cased in regcomp and regexec\, but I wanted to avoid that. Instead the mktables tables are set up to include the CAP MU as a character whose presence forbids the downgrading\, so the special casing is in mktables\, and not in the C code. (END) C:\perl519\src\t> ---------------------------------------------------------------------
I tried stepping locale.t but fresh_perl_is() blocked my debugging efforts a bit.
--------------------------------------------------------------------- my ($base\, $different\, $comma\, $difference); for ("C"\, @locales) { # prefer C for the base if available BEGIN { if($Config{d_setlocale}) { require locale; import locale; } } setlocale(LC_NUMERIC\, $_) or next; my $in = 4.2; # avoid any constant folding bugs if ((my $s = sprintf("%g"\, $in)) eq "4.2") { $base ||= $_; } else { $different ||= $_; $difference ||= $s; $comma ||= $_ if localeconv()->{decimal_point} eq '\,'; }
last if $base && $different && $comma; }
--------------------------------------------------------------------- sprintf did use "\," instead of "." occasionally.
here is the 1st fresh_perl_is\, ------------------------------------------------------------------------ 1..16 not ok 1 - no locales where LC_NUMERIC breaks # Failed test 1 - no locales where LC_NUMERIC breaks at ./test.pl line 954 # got undef # expected "" # PROG: # for (qw(Afrikaans Albanian American Basque Bulgarian C Croatian Czech Danish Dutch English Estonian Finnish Galician German Greek Hungarian Indonesian Italian Latvian Lithuanian Norwegian Polish Russian Serbian Slovak Slovenian Swedish Turkish Welsh af da en fi fy ga hu it la mo no no\@nynorsk ro ru sl)) { # use POSIX qw(locale_h); # use locale; # setlocale(LC_NUMERIC\, "$_") or next; # my $s = sprintf "%g %g"\, 3.1\, 3.1; # next if $s eq '3.1 3.1' || $s =~ /^(3.+1) \1$/; # print "$_ $s\n"; # } # STATUS: -1 ------------------------------------------------------------------------
I saved the tmp file and ran it manually\, ------------------------------------------------------------------------ C:\perl519\src\t>..\perl -I..\lib test1.pl
C:\perl519\src\t> ------------------------------------------------------------------------
next I tried a different test\, ------------------------------------------------------------------------ not ok 15 - POSIX::strtod() uses underlying locale # Failed test 15 - POSIX::strtod() uses underlying locale at ./test.pl line 954 # got undef # expected "1.5" # PROG: # use POSIX; # POSIX::setlocale(POSIX::LC_NUMERIC()\,"Albanian"); # my $one_point_5 = POSIX::strtod("1\,5"); # $one_point_5 =~ s/0+$//; # Remove any trailing zeros # print $one_point_5\, " # "; ------------------------------------------------------------------------- C:\perl519\src\t>..\perl -I..\lib test1.pl 1.5
C:\perl519\src\t>
-------------------------------------------------------------------------
stepping t/test.pl in runperl()\, $runperl is "C:\perl519\src\perl.exe "-I../lib" "tmp7516A" 2>&1"\, no outer quotes\, result of backticks is undef or empty string according to my watch window. Cwd of perl process is "C:\perl519\src\t\" when it does backticks. $? after backticks is -1.
Migrated from rt.perl.org#121106 (status was 'resolved')
Searchable as RT121106$