Perl / perl5

🐪 The Perl programming language
https://dev.perl.org/perl5/
Other
1.98k stars 559 forks source link

Undef loop while condition, loop code motion, and bad warning line number? 5.10-RC1 + 5.8.8 #9127

Open p5pRT opened 16 years ago

p5pRT commented 16 years ago

Migrated from rt.perl.org#47632 (status was 'open')

Searchable as RT47632$

p5pRT commented 16 years ago

From William.Ricker@FMR.COM

Created by a273121@fmr.com

Summary -

A uninitialized undef Value in   while($mightbeundef ... ){... results in warning that have strange line number\, in some cases.

User ID --

a273121@​fmr.com on Linux = William.Ricker@​FMR.com more generally   = is also known as use.perl.org/~n1vux \, Boston.pm type person

Version + BugLevel -

BOTH 5.10-RC1 (Linux) + 5.8.8/5.9.5 (Windows)\, possibly others.

"Medium" may be over-high for a misleading warning that 5.10 *has* improved. However\, this Bug applies to 5.8.8 as well\, where the better messages in 5.10 doesn't help figure out the bad line number\, so more serious on 5.8.8. I'd be happy with LOW on 5.10 and MEDIUM on 5.8.8.

Figured we should send it in during 5.10-RC1 so you have a chance to get the fix into 5.10 if it's easy\, since we can reproduce it in RC1.

Speculation -

This may be related to code-motion optimization in loop\, as it uses the line# of last statement of body as line# for control expression many lines before in source. (Simplest versions of code get correct line #.)

Background / detail -

A user adopting a complex non-CPAN published module (COBOLIO.pm) reported a strange uninitialized value warning for a line unrelated to any variable.

Inserting !defined in loop control expression of the loop of which the line numer given was last executable statement made the warning go away.

(This *will* [has been?] be reported to module author separately as a module bug. This report is about the *bad* line# on the warning.)

Using locally built 5.9.5 (Windows) or 5.10.0_RC1 (Linux) gives a better warning\, with name of variable\, great progress\, but still not right line number.

REPRODUCTION TESTCASE --

Our first attempt to remove all other bits of the module to make a simple test case failed; the simplest test case warning warned the "right" line. But we found something just simple enough.

Here's the simple reproduction that tricks it into appearing. The following should warn on use at line 12\, warns at 14.

[perl-5.10.0-RC1]$ cat bad2.pl #! bleedperl/bin/perl

use strict; use warnings;

our @​recLevel= (); our $currLevel=9; our $level=3;

sub foo {   my (@​elsewhere\, $c); # temp   while($currLevel > $level) { # 12   $c = $currLevel = pop @​recLevel; # 13   push @​elsewhere\, $c; # 14   } # 15

}

  foo();===EOF===

[perl-5.10.0-RC1]$ perl510 bad2.pl Use of uninitialized value $currLevel in numeric gt (>) at bad2.pl line 14.

Note there is no $currLevel in line #14. (If not using $c temp\, there would be and it would be MORE confusing!)

IMPACT UNDER 5.8.8 --

If any other uses of '>' occur in loop\, the 5.8.8 version of this message is

  Use of uninitialized value in numeric gt (>) at bad2.pl line 14.

Which is even more misleading. This confusion was such that our user downloaded the tarball for 5.9.5 to get the 5.10 better message\, which solved the conundrum. Hence I think this is long term more important for a 5.8.x maintenance release; if it is triaged to miss 5.10.0 and is fixed in 5.8.9 and ported to 5.10.1\, that's completely understandable\, we just want the p5p's to have the option.

Perl Info ``` Flags: category=core severity=medium Site configuration information for perl 5.10.0: Configured by a273121 at Tue Nov 20 15:16:33 EST 2007. Summary of my perl5 (revision 5 version 10 subversion 0) configuration: Platform: osname=linux, osvers=2.6.9-55.elsmp, archname=x86_64-linux uname='linux lmro273 2.6.9-55.elsmp #1 smp fri apr 20 16:36:54 edt 2007 x86_64 x86_64 x86_64 gnulinux ' config_args='-des -Dprefix=/export/home/a273121/perl-5.10.0-RC1/bleedperl -Dusesitecustomize -Duserelocatableinc' hint=recommended, useposix=true, d_sigaction=define useithreads=undef, usemultiplicity=undef useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef use64bitint=define, use64bitall=define, uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cc', ccflags ='-fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-O2', cppflags='-fno-strict-aliasing -pipe -I/usr/local/include' ccversion='', gccversion='3.4.6 20060404 (Red Hat 3.4.6-8)', gccosandvers='' intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16 ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='cc', ldflags =' -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib /lib64 /usr/lib64 /usr/local/lib64 libs=-lnsl -ldl -lm -lcrypt -lutil -lc perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc libc=/lib/libc-2.3.4.so, so=so, useshrplib=false, libperl=libperl.a gnulibc_version='2.3.4' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E' cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib' Locally applied patches: RC1 @INC for perl 5.10.0: /export/home/a273121/perl-5.10.0-RC1/bleedperl/lib/5.10.0/x86_64-linux /export/home/a273121/perl-5.10.0-RC1/bleedperl/lib/5.10.0 /export/home/a273121/perl-5.10.0-RC1/bleedperl/lib/site_perl/5.10.0/x86_ 64-linux /export/home/a273121/perl-5.10.0-RC1/bleedperl/lib/site_perl/5.10.0 . Environment for perl 5.10.0: HOME=/export/home/a273121 LANG=en_US.UTF-8 LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/fisc/uts/bin:/usr/X 11R6/bin PERL_BADLANG (unset) SHELL=/bin/ksh ```
p5pRT commented 16 years ago

From @schwern

Ricker\, William (via RT) wrote​:

A uninitialized undef Value in while($mightbeundef ... ){... results in warning that have strange line number\, in some cases.

Thanks for your very through report.

This is like a consequence of a very old issue that nobody's quite figured out how to resolve. Perl tends to get a little confused with regard to getting the line number right around a block condition.

However\, I notice that 5.6.2 got this right. Andreas\, could you apply your binary patch searcher to this problem?

-- ROCKS FALL! EVERYONE DIES!   http​://www.somethingpositive.net/sp05032002.shtml

p5pRT commented 16 years ago

The RT System itself - Status changed from 'new' to 'open'

p5pRT commented 16 years ago

From @andk

On Wed\, 21 Nov 2007 13​:32​:58 -0800\, Michael G Schwern \schwern@​pobox\.com said​:

  > Ricker\, William (via RT) wrote​:

A uninitialized undef Value in while($mightbeundef ... ){... results in warning that have strange line number\, in some cases.

  > Thanks for your very through report.

  > This is like a consequence of a very old issue that nobody's quite figured out   > how to resolve. Perl tends to get a little confused with regard to getting   > the line number right around a block condition.

  > However\, I notice that 5.6.2 got this right. Andreas\, could you apply your   > binary patch searcher to this problem?

19610 looks very suspicious​:

Change 19610 by jhi@​kosh on 2003/05/24 06​:42​:52

  Subject​: [PATCH #2] Re​: [perl #22181] goto undefines my() variables   From​: Dave Mitchell \davem@​fdgroup\.com   Date​: Thu\, 22 May 2003 10​:13​:19 +0100   Message-ID​: \20030522091319\.GA4568@​fdgroup\.com  
  Subject​: Re​: [PATCH #2] Re​: [perl #22181] goto undefines my() variables   From​: Dave Mitchell \davem@​fdgroup\.com   Date​: Fri\, 23 May 2003 17​:09​:44 +0100   Message-ID​: \20030523160944\.GC9194@​fdgroup\.com

  ----Program----

  use strict;   use warnings;

  our @​recLevel= ();   our $currLevel=9;   our $level=3;

  sub foo {   my (@​elsewhere\, $c); # temp   while($currLevel > $level) { # 12   $c = $currLevel = pop @​recLevel; # 13   push @​elsewhere\, $c; # 14   } # 15

  }   $SIG{__WARN__} = sub { my $warn = shift; print $warn =~ /line (\d+)/ ? "->$1\<-\n" : $warn; };

  foo();

  ----Output of .../poKIX8P/perl-5.8.0@​19608/bin/perl----   ->12\<-

  ----EOF ($?='0')----   ----Output of .../ppLsGGj/perl-5.8.0@​19610/bin/perl----   ->14\<-

  ----EOF ($?='0')----

-- andreas

p5pRT commented 16 years ago

From @iabyn

On Fri\, Nov 23\, 2007 at 10​:45​:32PM +0100\, Andreas J. Koenig wrote​:

On Wed\, 21 Nov 2007 13​:32​:58 -0800\, Michael G Schwern \schwern@&#8203;pobox\.com said​:

Ricker\, William (via RT) wrote​:

A uninitialized undef Value in while($mightbeundef ... ){... results in warning that have strange line number\, in some cases.

Thanks for your very through report.

This is like a consequence of a very old issue that nobody's quite figured out how to resolve. Perl tends to get a little confused with regard to getting the line number right around a block condition.

However\, I notice that 5.6.2 got this right. Andreas\, could you apply your binary patch searcher to this problem?

19610 looks very suspicious​:

This is mainly a red herring. The basic problem is that line numbers are only recorded once per statement rather than once per op. This gives rise to quite a few line-numbering bugs (especially with if/then and while loops). Change 19610 just happened to alter when and where nextstate gets executed as as side-effect of fixing a bug.

The real fix is to record the line number in every op\, at the cost of making ops bigger. Something to do post 5.10.0 ...

-- Never work with children\, animals\, or actors.

p5pRT commented 16 years ago

From @nwc10

On Sat\, Nov 24\, 2007 at 01​:28​:31AM +0000\, Dave Mitchell wrote​:

The real fix is to record the line number in every op\, at the cost of making ops bigger. Something to do post 5.10.0 ...

Or at least in every op where the line number changed.

Could one get away with a null op in the same family as nextstate that would be able to record that line number change without getting in the way of execution?

Nicholas Clark

p5pRT commented 16 years ago

From William.Ricker@FMR.COM

1) Obviously I intend a newline (or #) after foo(); and before ===EOF=== annotation\,   perhaps I should use __END__ ...   (VI famously omits NL at EOF\, and I pasted ... oopsie)

2) Same result with perl-5.10-RC2 [although RC2 was harder to build relocatable]   Use of uninitialized value $currLevel in numeric gt (>) at bad2.pl line 14.   (should be @​ 12)

  while($currLevel > $level) { # 12   $c = $currLevel = pop @​recLevel; # 13   push @​elsewhere\, $c; # 14   } # 15

3) Before someone asks\, Yes\, 'make test' succeeds on my RC1 & RC2 Linux builds.

4) I finally found an AIX5.3 development system with the newer VAC 6 and space on $HOME.   I get same result there too\, with RC2. ('make test' ok)

William Ricker Principal (swe)\, Architecture Services Fidelity Investments / FPCMS / Systems 617-563-0648 M/S Z1E William.Ricker@​FMR.com 1141547 Skytel http​://www.linkedin.com/in/n1vux

p5pRT commented 16 years ago

From William.Ricker@FMR.COM

Michael\, Andreas\, Dave\, Nick --

Thanks for the great explanation.

We are not surprised that it's not an easy fix for 5.10.0; that's pretty much what we expected from dark-ages compiler theory dimly remembered. But the ideas for alternative future solutions are fascinating\, and is more hopeful than we'd dared hope.

I have not come up with an example that is not erroneous; but then\, warnings are to help the erroneous. At least 5.10 tells us WHAT variable became undef\, even if not the right line sometimes. The module that evoked this warning should probably have a   last unless $currLevel; # hit bottom early\, so done inserted at the bottom or   defined($currLevel) and in the condition at the top\, to avoid runtime blather to STDERR and scaring module users. We _have_ reported this\, but haven't submitted a patch since we're not Absolutely certain that that is the correct fix to the module.

[for BCC​:'s - see discussion at http​://rt.perl.org/rt3/Public/Bug/Display.html?id=47632 ]

Cheers\,

BILL in Boston

William Ricker Principal (swe)\, Architecture Services Fidelity Investments / FPCMS / Systems +1 617-563-0648 William.Ricker@​FMR.com http​://www.linkedin.com/in/n1vux Aka http​://use.perl.org/~n1vux   + http​://boston.pm.org/kwiki/index.cgi?BillRicker   + bill.n1vux@​gmail.com and maybe even uunet!world!wdr