Bleadperl v5.19.0-570-gb127e37 breaks PERLER/JavaScript-Dumper-0.011.tar.gz

p5pRT commented 10 years ago

Migrated from rt.perl.org#121013 (status was 'resolved')

Searchable as RT121013$

p5pRT commented 10 years ago

From @andk

git

commit b127e37e51c21b0a36755dcd19811be931a03d83 Author: Karl Williamson \public@khwilliamson\.com Date: Mon May 13 07:35:35 2013 -0600

PATCH: [perl #108378] [perl #115800]

diagnostics

http://www.cpantesters.org/cpan/report/d2dd3370-d826-11e2-bc6f-95dabcbf7481

Thanks to Slaven Rezić for finding this.

perl -V

Summary of my perl5 (revision 5 version 19 subversion 1) configuration: Commit id: b127e37e51c21b0a36755dcd19811be931a03d83 Platform: osname=linux\, osvers=3.10-3-amd64\, archname=x86_64-linux-thread-multi-ld uname='linux k83 3.10-3-amd64 #1 smp debian 3.10.11-1 (2013-09-10) x86_64 gnulinux ' config_args='-Dprefix=/home/sand/src/perl/repoperls/installed-perls/perl/v5.19.0-570-gb127e37/a2da -Dmyhostname=k83 -Dinstallusrbinperl=n -Uversiononly -Dusedevel -des -Ui_db -Duseithreads -Duselongdouble -DDEBUGGING=-g' hint=recommended\, useposix=true\, d_sigaction=define useithreads=define\, usemultiplicity=define useperlio=define\, d_sfio=undef\, uselargefiles=define\, usesocks=undef use64bitint=define\, use64bitall=define\, uselongdouble=define usemymalloc=n\, bincompat5005=undef Compiler: cc='cc'\, ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'\,
optimize='-O2 -g'\, cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include' ccversion=''\, gccversion='4.8.2'\, gccosandvers='' intsize=4\, longsize=8\, ptrsize=8\, doublesize=8\, byteorder=12345678 d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=16 ivtype='long'\, ivsize=8\, nvtype='long double'\, nvsize=16\, Off_t='off_t'\, lseeksize=8 alignbytes=16\, prototype=define Linker and Libraries: ld='cc'\, ldflags =' -fstack-protector -L/usr/local/lib' libpth=/usr/local/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /usr/lib libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc -lgdbm_compat perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc libc=\, so=so\, useshrplib=false\, libperl=libperl.a gnulibc_version='2.17' Dynamic Linking: dlsrc=dl_dlopen.xs\, dlext=so\, d_dlsymun=undef\, ccdlflags='-Wl\,-E' cccdlflags='-fPIC'\, lddlflags='-shared -O2 -g -L/usr/local/lib -fstack-protector'

Characteristics of this binary (from libperl): Compile-time options: HAS_TIMES MULTIPLICITY PERLIO_LAYERS PERL_DONT_CREATE_GVSV PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP PERL_NEW_COPY_ON_WRITE PERL_PRESERVE_IVUV PERL_USE_DEVEL USE_64_BIT_ALL USE_64_BIT_INT USE_ITHREADS USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_LONG_DOUBLE USE_PERLIO USE_PERL_ATOF USE_REENTRANT_API Built under linux Compiled at Jan 16 2014 06:43:25 @INC: /home/sand/src/perl/repoperls/installed-perls/perl/v5.19.0-570-gb127e37/a2da/lib/site_perl/5.19.1/x86_64-linux-thread-multi-ld /home/sand/src/perl/repoperls/installed-perls/perl/v5.19.0-570-gb127e37/a2da/lib/site_perl/5.19.1 /home/sand/src/perl/repoperls/installed-perls/perl/v5.19.0-570-gb127e37/a2da/lib/5.19.1/x86_64-linux-thread-multi-ld /home/sand/src/perl/repoperls/installed-perls/perl/v5.19.0-570-gb127e37/a2da/lib/5.19.1 .

-- andreas

p5pRT commented 10 years ago

From @khwilliamson

Here's the email we got for one ticket that would be nice to add to https://rt-archive.perl.org/perl5/Ticket/Display.html?id=116923 for 5.20. After 5.20 ships\, we could manually add a ticket for 5.22 and do something so that such emails point to that ticket.

It occurs to me that these emails may be machine generated\, and if so\, we could request their originator to change the code to add them to the current ticket for blockers\, if that were feasible.

-------- Original Message -------- From: - Thu Jan 16 00:06:33 2014 X-Account-Key: account2 X-UIDL: 0002896e4ad3b7bf X-Mozilla-Status: 0001 X-Mozilla-Status2: 00000000 X-Mozilla-Keys: Received: from x6.develooper.com (x6.develooper.com [207.171.7.86]) by net.indra.com (8.14.7/8.14.7) with ESMTP id s0G6utEQ071871 for \public@khwilliamson\.com; Wed\, 15 Jan 2014 23:56:57 -0700 (MST) (envelope-from perl5-porters-return-211508-public=khwilliamson.com@perl.org) Received: from lists-nntp.develooper.com (localhost.localdomain [127.0.0.1]) by x6.develooper.com (Postfix) with SMTP id D80822A4E for \public@khwilliamson\.com; Wed\, 15 Jan 2014 22:56:46 -0800 (PST) Received: (qmail 4640 invoked by uid 514); 16 Jan 2014 06:56:45 -0000 Mailing-List: contact perl5-porters-help@perl.org; run by ezmlm Precedence: bulk list-help: \mailto:perl5\-porters\-help@perl\.org list-unsubscribe: \mailto:perl5\-porters\-unsubscribe@perl\.org list-post: \mailto:perl5\-porters@perl\.org X-List-Archive: \<http://nntp.perl.org/group/perl.perl5.porters/211508> List-Id: \<perl5-porters.perl.org> Delivered-To: mailing list perl5-porters@perl.org Received: (qmail 4628 invoked from network); 16 Jan 2014 06:56:45 -0000 Delivered-To: perl5-porters@perl.org X-Spam-Status: No\, hits=-5.1 required=8.0 tests=BAYES_00\,PERLBUG_CONF\,SPF_NEUTRAL X-Spam-Check-By: la.mx.develooper.com Delivered-To: rt-perl5-testers@x1.develooper.com Mail-From: perlbug-followup@perl.org Wed Jan 15 22:56:40 2014 Delivered-To: bugs-perl5-testers@rt.perl.org From: (Andreas J. Koenig) (via RT) \perlbug\-followup@perl\.org X-RT-NewTicket: yes To: bugs-bitbucket@rt.perl.org Mail-Followup-To: perl5-porters@perl.org Reply-To: perl5-porters@perl.org X-RT-Will-Also-CC: andreas.koenig.7os6VVqR@franz.ak.mind.de\, Subject: [perl #121013] Bleadperl v5.19.0-570-gb127e37 breaks PERLER/JavaScript-Dumper-0.011.tar.gz In-Reply-To: \87ob3ctnxr\.fsf@k85\.linux\.bogus References: \RT\-Ticket\-121013@perl\.org \87ob3ctnxr\.fsf@k85\.linux\.bogus Message-ID: \rt\-4\.0\.18\-29970\-1389855395\-1178\.121013\-75\-0@perl\.org X-RT-Loop-Prevention: perl RT-Ticket: perl #121013 Managed-BY: RT 4.0.18 (http://www.bestpractical.com/rt/) RT-Originator: andreas.koenig.7os6VVqR@franz.ak.mind.de MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-RT-Original-Encoding: utf-8 Date: Wed\, 15 Jan 2014 22:56:35 -0800 Resent-To: perl5-porters@perl.org X-Old-Spam-Check-By: la.mx.develooper.com X-Old-Spam-Status: No\, hits=-6.2 required=8.0 tests=BAYES_00\,PERLBUG_CONF\,RP_MATCHES_RCVD\,SPF_PASS Resent-Message-Id: \20140116065646\.D80822A4E@x6\.develooper\.com Resent-Date: Wed\, 15 Jan 2014 22:56:46 -0800 (PST) Resent-From: perl5-porters-return-211508-public=khwilliamson.com@perl.org X-DCC-indra.com-Metrics: net.indra.com; whitelist X-Likely-Spam: true X-Likely-Spam: true

# New Ticket Created by (Andreas J. Koenig) # Please include the string: [perl #121013] # in the subject line of all future correspondence about this issue. # \<URL: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=121013 >

git

commit b127e37e51c21b0a36755dcd19811be931a03d83 Author: Karl Williamson \public@khwilliamson\.com Date: Mon May 13 07:35:35 2013 -0600

PATCH: [perl #108378] [perl #115800]

diagnostics

http://www.cpantesters.org/cpan/report/d2dd3370-d826-11e2-bc6f-95dabcbf7481

Thanks to Slaven Rezić for finding this.

perl -V

Summary of my perl5 (revision 5 version 19 subversion 1) configuration: Commit id: b127e37e51c21b0a36755dcd19811be931a03d83 Platform: osname=linux\, osvers=3.10-3-amd64\, archname=x86_64-linux-thread-multi-ld uname='linux k83 3.10-3-amd64 #1 smp debian 3.10.11-1 (2013-09-10) x86_64 gnulinux '

config_args='-Dprefix=/home/sand/src/perl/repoperls/installed-perls/perl/v5.19.0-570-gb127e37/a2da -Dmyhostname=k83 -Dinstallusrbinperl=n -Uversiononly -Dusedevel -des -Ui_db -Duseithreads -Duselongdouble -DDEBUGGING=-g' hint=recommended\, useposix=true\, d_sigaction=define useithreads=define\, usemultiplicity=define useperlio=define\, d_sfio=undef\, uselargefiles=define\, usesocks=undef use64bitint=define\, use64bitall=define\, uselongdouble=define usemymalloc=n\, bincompat5005=undef Compiler: cc='cc'\, ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'\, optimize='-O2 -g'\, cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include' ccversion=''\, gccversion='4.8.2'\, gccosandvers='' intsize=4\, longsize=8\, ptrsize=8\, doublesize=8\, byteorder=12345678 d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=16 ivtype='long'\, ivsize=8\, nvtype='long double'\, nvsize=16\, Off_t='off_t'\, lseeksize=8 alignbytes=16\, prototype=define Linker and Libraries: ld='cc'\, ldflags =' -fstack-protector -L/usr/local/lib' libpth=/usr/local/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /usr/lib libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc -lgdbm_compat perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc libc=\, so=so\, useshrplib=false\, libperl=libperl.a gnulibc_version='2.17' Dynamic Linking: dlsrc=dl_dlopen.xs\, dlext=so\, d_dlsymun=undef\, ccdlflags='-Wl\,-E' cccdlflags='-fPIC'\, lddlflags='-shared -O2 -g -L/usr/local/lib -fstack-protector'

Characteristics of this binary (from libperl): Compile-time options: HAS_TIMES MULTIPLICITY PERLIO_LAYERS PERL_DONT_CREATE_GVSV PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP PERL_NEW_COPY_ON_WRITE PERL_PRESERVE_IVUV PERL_USE_DEVEL USE_64_BIT_ALL USE_64_BIT_INT USE_ITHREADS USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_LONG_DOUBLE USE_PERLIO USE_PERL_ATOF USE_REENTRANT_API Built under linux Compiled at Jan 16 2014 06:43:25 @INC:

/home/sand/src/perl/repoperls/installed-perls/perl/v5.19.0-570-gb127e37/a2da/lib/site_perl/5.19.1/x86_64-linux-thread-multi-ld

/home/sand/src/perl/repoperls/installed-perls/perl/v5.19.0-570-gb127e37/a2da/lib/site_perl/5.19.1

/home/sand/src/perl/repoperls/installed-perls/perl/v5.19.0-570-gb127e37/a2da/lib/5.19.1/x86_64-linux-thread-multi-ld

/home/sand/src/perl/repoperls/installed-perls/perl/v5.19.0-570-gb127e37/a2da/lib/5.19.1 .

-- andreas

p5pRT commented 10 years ago

The RT System itself - Status changed from 'new' to 'open'

p5pRT commented 10 years ago

From @khwilliamson

On Wed Jan 15 22:56:35 2014\, andreas.koenig.7os6VVqR@franz.ak.mind.de wrote:

git --- commit b127e37e51c21b0a36755dcd19811be931a03d83 Author: Karl Williamson \public@khwilliamson\.com Date: Mon May 13 07:35:35 2013 -0600

PATCH: [perl #108378] [perl #115800]

diagnostics ----------- http://www.cpantesters.org/cpan/report/d2dd3370-d826-11e2-bc6f- 95dabcbf7481

Thanks to Slaven Rezić for finding this.

The test that fails in 5.19 is expecting a floating point number to be returned by JSON to be enclosed in double quotes\, which it used to be\, but after the blamed commit\, no longer is. I have tracked this down to the following statement in JSON::PP (JSON:XS has a similar logic):

return $value # as is if $flags & ( B::SVp_IOK | B::SVp_NOK ) and !( $flags & B::SVp_POK ); # SvTYPE is IV or NV?

The blamed commit changed things so that some scalars that are NVs no longer are also PVs. In the past this statement would not return and code further down would enclose it in double quotes\, hence the old behavior\, and the new.

But the logic here seems wrong to me. I don't understand the possible nuances and gotchas\, but it seems to me that if something is an NV\, it should be returned as an NV\, without regard to if there is a stringified version available or not\, which is what SVp_POK indicates.

My guess is that the tests for POK are to detect cases where something has a numeric value\, but its string is different from that value and needs to be preserved\, such as perhaps "00" (I don't know the internals here\, so am just guessing.) But there is no restriction on somebody's deciding to make a numeric scalar POK. It can be done for good reasons\, and it could be done for invalid ones. The modules shouldn't depend on this.

(In this case\, prior to the blamed commit\, a floating point number was made POK when its string form was needed\, in order to avoid having to recalculate it each time. But this was found to create bugs where the decimal point varies in a program between\, say\, a comma and a dot because of locale differences. Making it POK cemented the decimal point to what it was at the time it was first stringified\, not what it should be the next time a stringified version is needed. So it isn't made POK and is recalculated each time)

I think that both JSON::PP and JSON::XS should add logic to better detect whether something's string is different from its numeric value

-- Karl Williamson

p5pRT commented 10 years ago

From schmorp@schmorp.de

On Wed\, Mar 05\, 2014 at 11:09:47AM -0800\, Karl Williamson via RT \perlbug\-followup@perl\.org wrote:

The test that fails in 5.19 is expecting a floating point number to be returned by JSON to be enclosed in double quotes\, which it used to be\, but after the blamed commit\, no longer is. I have tracked this down to the following statement in JSON::PP (JSON:XS has a similar logic):

Without seeing the test code nor the patch (both weren't easily available)\, I can only make some general comments\, however\, there is a good chance that the test itself is buggy.

Perl scalars are (for the most part) typeless - scalars are not NVs or PVs\, not even internally.

JSON::XS (and JSON::PP\, which copied the logic) attempts the impossible "and guesses" the type - the rules for that are described in the section "MAPPING" in the JSON::XS documentation.

The short version is that the last use/modification counts\, and the string form is preferred when in doubt\, so if you printed your NV\, or stringified it\, it will serialise as a string.

But the logic here seems wrong to me. I don't understand the possible nuances and gotchas\, but it seems to me that if something is an NV\, it should be returned as an NV\, without regard to if there is a stringified version available or not\, which is what SVp_POK indicates.

In your case\, this is true\, in other cases\, this isn't. If you want to make sure that your number is serialised as number\, then you need to +=0 it for example.

The the treatment of "types" varies considerably within perl versions\, and the test as-is is the one that works reliably for perl versions since 5.8\, and errs on the safe side.

My guess is that the tests for POK are to detect cases where something has a numeric value\, but its string is different from that value and needs to be preserved\, such as perhaps "00" (I don't know the internals here\, so am just guessing.) But there is no restriction on somebody's deciding to make a numeric scalar POK. It can be done for good reasons\, and it could be done for invalid ones. The modules shouldn't depend on this.

The test si simply there because in JSON\, the number 5 is different than the string "5"\, while in Perl\, there is no semantic difference between them.

(In this case\, prior to the blamed commit\, a floating point number was made POK when its string form was needed\, in order to avoid having to recalculate it each time. But this was found to create bugs where the decimal point varies in a program between\, say\, a comma and a dot because of locale differences.

The locale changes recently introduced in 5.19 cause a great deal of breakage (probably again a case where the new p5ps don't care for backwards compatibility). I hope these are sorted out before it gets released.

I think that both JSON::PP and JSON::XS should add logic to better detect whether something's string is different from its numeric value

The problem is that it's not clear what to do in that case. JSON::XS has to decide on either string or number.

If the += 0 "trick" no longer works because perl stringifies the NV\, then we are indeed in trouble\, as that makes it impossible to even semi-reliably guess types.

However\, such a change would break an enourmous number of programs (most sql drivers use a similar form of guessing)\, and one would hope that p5p would not introduce such a big change (although in the past\, they didn't seem to be reluctant to break things for no demonstrable gain).

-- The choice of a Deliantra\, the free code+content MORPG -----==- _GNU_ http://www.deliantra.net ----==-- _ generation ---==---(_)__ __ ____ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / schmorp@schmorp.de -=====/_/_//_/\_\,_/ /_/\_\

p5pRT commented 10 years ago

From @khwilliamson

On 3/5/2014 10:06 PM\, Marc Lehmann wrote:

The short version is that the last use/modification counts\, and the string form is preferred when in doubt\, so if you printed your NV\, or stringified it\, it will serialise as a string.

But the logic here seems wrong to me. I don't understand the possible nuances and gotchas\, but it seems to me that if something is an NV\, it should be returned as an NV\, without regard to if there is a stringified version available or not\, which is what SVp_POK indicates. In your case\, this is true\, in other cases\, this isn't. If you want to make sure that your number is serialised as number\, then you need to +=0 it for example.

The the treatment of "types" varies considerably within perl versions\, and the test as-is is the one that works reliably for perl versions since 5.8\, and errs on the safe side.

What if the code in JOSN::PP instead did something like this:

if ($flags & ( B::SVp_IOK | B::SVp_NOK ) { # SvTYPE is IV or NV? return $value if ! $flags & B::SVp_POK; # as is my $numeric = $value + 0; return $value if "$numeric" eq $value; # stringified is identical to numeric }

Would that also work reliably? And insulate the code from us cretins on p5p?

p5pRT commented 10 years ago

From @khwilliamson

I submitted a patch to the distribution https://rt.cpan.org/Public/Bug/Display.html?id=95230

so am resolving this core ticket -- Karl Williamson

p5pRT commented 10 years ago

@khwilliamson - Status changed from 'open' to 'resolved'

Perl / perl5