bytecode fails with use subs;

[p5pRT]

Migrated from rt.perl.org#3917 (status was 'resolved')

Searchable as RT3917$

[p5pRT]

From @nwc10

Created by @nwc10

using perlcc -B to generate bytecode\, I find that use subs ...; in generates bytecode that won't run

Simplest test case is​:

#!/usr/local/bin/perl -w use subs 'test';

&test; sub test {   print "bing\n"; }

nick@​Bagpuss [Bytecode]$ perlcc.pl -B -o test test.pl nick@​Bagpuss [Bytecode]$ perl test Undefined subroutine &main​::test called at test.pl line 4.

If I remove the use subs line then it all works. This stops me using perlcc to byte compile itself. (seemed a suitable dogfood challenge) I can't work out what's causing the problem - I've tried emulating use subs with *{"main​::test"} = \&{"main​::test"}; instead of the use subs; line but that makes new problems​:

nick@​Bagpuss [Bytecode]$ cat test2.pl #!/usr/local/bin/perl -w

package subs; *{"main​::test"} = \&{"main​::test"}; package main;

&test; sub test {   print "bing\n"; } nick@​Bagpuss [Bytecode]$ nasty ./perlcc.pl -B -o test2 test2.pl nick@​Bagpuss [Bytecode]$ perl test2 bing Segmentation fault

so I don't think this is helping. I've checked with a packages; statement\, and this isn't limited to main.

If I use the -S option to the B​::Bytecode backend\, then

with 'use subs' I get lines   newpv "test\000"   gv_fetchpv "main​::test"

without 'use subs' I get   gv_fetchpv "main​::test"   newpv "bing\n\000"

so it seems that with the 'use subs' the backend is failing to traverse the op tree. I don't know the significance of this. Hopefully the significance will be obvious to someone else

Nicholas Clark

Perl Info

``` Flags: category=library severity=medium Site configuration information for perl v5.7.0: Configured by nick at Sun Aug 27 17:26:14 BST 2000. Summary of my perl5 (revision 5.0 version 7 subversion 0) configuration: Platform: osname=linux, osvers=2.2.16-rmk2, archname=armv4l-linux uname='linux bagpuss.unfortu.net 2.2.16-rmk2 #8 thu jul 20 12:24:36 bst 2000 armv4l unknown ' config_args='' hint=previous, useposix=true, d_sigaction=define usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef useperlio=undef d_sfio=undef uselargefiles=define use64bitint=undef use64bitall=undef uselongdouble=undef usesocks=undef Compiler: cc='cc', optimize='-g', gccversion=2.95.2 20000220 (Debian GNU/Linux), gccosandvers= cppflags='-fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64' ccflags ='-fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64' stdchar='char', d_stdstdio=define, usevfork=false intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=8 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=4, usemymalloc=n, prototype=define Linker and Libraries: ld='cc', ldflags =' -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib libs=-lsfio -lnsl -lndbm -ldb -ldl -lm -lc -lposix -lcrypt -lutil libc=/lib/libc-2.1.3.so, so=so, useshrplib=false, libperl=libperl.a Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic' cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib' Locally applied patches: SUIDMAIL - fixes for suidperl security DEVEL6849 @INC for perl v5.7.0: /usr/local/lib/perl5/5.7.0/armv4l-linux /usr/local/lib/perl5/5.7.0 /usr/local/lib/perl5/site_perl/5.7.0/armv4l-linux /usr/local/lib/perl5/site_perl/5.7.0 /usr/local/lib/perl5/site_perl . Environment for perl v5.7.0: HOME=/home/nick LANG (unset) LANGUAGE (unset) LC_CTYPE=en_GB.ISO-8859-1 LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/home/nick/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/games:/sbin:/usr/sbin:/usr/local/sbin PERL_BADLANG (unset) SHELL=/bin/bash ```

[p5pRT]

From [Unknown Contact. See original ticket]

--- Nicholas Clark \nick@​ccl4\.org wrote​:

This is a bug report for perl from nick@​talking.bollo.cx\, generated with the help of perlbug 1.32 running under perl v5.7.0.

---

[Please enter your report here]

using perlcc -B to generate bytecode\, I find that use subs ...; in generates bytecode that won't run

Simplest test case is​:

#!/usr/local/bin/perl -w use subs 'test';

&test; sub test { print "bing\n"; }

nick@​Bagpuss [Bytecode]$ perlcc.pl -B -o test test.pl nick@​Bagpuss [Bytecode]$ perl test Undefined subroutine &main​::test called at test.pl line 4.

If I remove the use subs line then it all works. This stops me using perlcc to byte compile itself. (seemed a suitable dogfood challenge) I can't work out what's causing the problem - I've tried emulating use subs with *{"main​::test"} = \&{"main​::test"}; instead of the use subs; line but that makes new problems​:

nick@​Bagpuss [Bytecode]$ cat test2.pl #!/usr/local/bin/perl -w

package subs; *{"main​::test"} = \&{"main​::test"}; package main;

&test; sub test { print "bing\n"; } nick@​Bagpuss [Bytecode]$ nasty ./perlcc.pl -B -o test2 test2.pl nick@​Bagpuss [Bytecode]$ perl test2 bing Segmentation fault

so I don't think this is helping. I've checked with a packages; statement\, and this isn't limited to main.

If I use the -S option to the B​::Bytecode backend\, then

with 'use subs' I get lines newpv "test\000" gv_fetchpv "main​::test"

without 'use subs' I get gv_fetchpv "main​::test" newpv "bing\n\000"

so it seems that with the 'use subs' the backend is failing to traverse the op tree. I don't know the significance of this. Hopefully the significance will be obvious to someone else

The issue is that B​::Bytecode doesn't save subs marked as GVf_IMPORTED_CV in their GV\, under the belief that (since it recreates 'use Foo' at runtime) these subs will be recreated then. You might try directly invoking B​::Bytecode; try

perl -MO=Bytecode\,-O2\,-Pmain\,-Psubs -e 'use subs "foo"; foo(); sub foo { print "bar\n"; }'

However\, I have some doubts as to whether this will work (the code for sub foo still might not get saved). Try it and see :)

-- BKS

Nicholas Clark

[Please do not change anything below this line]

---

--- Flags​: category=library severity=medium --- Site configuration information for perl v5.7.0​:

Configured by nick at Sun Aug 27 17​:26​:14 BST 2000.

Summary of my perl5 (revision 5.0 version 7 subversion 0) configuration​: Platform​: osname=linux\, osvers=2.2.16-rmk2\, archname=armv4l-linux uname='linux bagpuss.unfortu.net 2.2.16-rmk2 #8 thu jul 20 12​:24​:36 bst 2000 armv4l unknown ' config_args='' hint=previous\, useposix=true\, d_sigaction=define usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef useperlio=undef d_sfio=undef uselargefiles=define use64bitint=undef use64bitall=undef uselongdouble=undef usesocks=undef Compiler​: cc='cc'\, optimize='-g'\, gccversion=2.95.2 20000220 (Debian GNU/Linux)\, gccosandvers= cppflags='-fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64' ccflags ='-fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64' stdchar='char'\, d_stdstdio=define\, usevfork=false intsize=4\, longsize=4\, ptrsize=4\, doublesize=8\, byteorder=1234 d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=8 ivtype='long'\, ivsize=4\, nvtype='double'\, nvsize=8\, Off_t='off_t'\, lseeksize=8 alignbytes=4\, usemymalloc=n\, prototype=define Linker and Libraries​: ld='cc'\, ldflags =' -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib libs=-lsfio -lnsl -lndbm -ldb -ldl -lm -lc -lposix -lcrypt -lutil libc=/lib/libc-2.1.3.so\, so=so\, useshrplib=false\, libperl=libperl.a Dynamic Linking​: dlsrc=dl_dlopen.xs\, dlext=so\, d_dlsymun=undef\, ccdlflags='-rdynamic' cccdlflags='-fpic'\, lddlflags='-shared -L/usr/local/lib'

Locally applied patches​: SUIDMAIL - fixes for suidperl security DEVEL6849

--- @​INC for perl v5.7.0​: /usr/local/lib/perl5/5.7.0/armv4l-linux /usr/local/lib/perl5/5.7.0 /usr/local/lib/perl5/site_perl/5.7.0/armv4l-linux /usr/local/lib/perl5/site_perl/5.7.0 /usr/local/lib/perl5/site_perl .

--- Environment for perl v5.7.0​: HOME=/home/nick LANG (unset) LANGUAGE (unset) LC_CTYPE=en_GB.ISO-8859-1 LD_LIBRARY_PATH (unset) LOGDIR (unset)

PATH=/home/nick/bin​:/usr/local/bin​:/usr/bin​:/bin​:/usr/bin/X11​:/usr/games​:/sbin​:/usr/sbin​:/usr/local/sbin PERL_BADLANG (unset) SHELL=/bin/bash

__________________________________________________ Do You Yahoo!? Yahoo! Mail - Free email you can access from anywhere! http​://mail.yahoo.com/

[p5pRT]

From @nwc10

On Tue\, Aug 29\, 2000 at 05​:23​:15AM -0700\, Benjamin Stuhl wrote​:

The issue is that B​::Bytecode doesn't save subs marked as GVf_IMPORTED_CV in their GV\, under the belief that (since it recreates 'use Foo' at runtime) these subs will be recreated then. You might try directly invoking B​::Bytecode; try

perl -MO=Bytecode\,-O2\,-Pmain\,-Psubs -e 'use subs "foo"; foo(); sub foo { print "bar\n"; }'

However\, I have some doubts as to whether this will work (the code for sub foo still might not get saved). Try it and see :)

nick@​fruitbat [bleadperl]$ echo use ByteLoader\; >bcode nick@​fruitbat [bleadperl]$ perl -MO=Bytecode\,-O2\,-Pmain\,-Psubs -e 'use subs "foo";
foo(); sub foo { print "bar\n"; }' >>bcode -e syntax OK nick@​fruitbat [bleadperl]$ perl bcode Undefined subroutine &main​::foo called at -e line 2.

Sadly not. Where do we go from here? Fiddle use subs; to generate different flags in the parent namespace?

Nicholas Clark

[p5pRT]

From [Unknown Contact. See original ticket]

--- Nicholas Clark \nick@​ccl4\.org wrote​:

On Tue\, Aug 29\, 2000 at 05​:23​:15AM -0700\, Benjamin Stuhl wrote​:

The issue is that B​::Bytecode doesn't save subs marked as GVf_IMPORTED_CV in their GV\, under the belief that (since it recreates 'use Foo' at runtime) these subs will be recreated then. You might try directly invoking B​::Bytecode; try

perl -MO=Bytecode\,-O2\,-Pmain\,-Psubs -e 'use subs "foo"; foo(); sub foo { print "bar\n"; }'

However\, I have some doubts as to whether this will work (the code for sub foo still might not get saved). Try it and see :)

nick@​fruitbat [bleadperl]$ echo use ByteLoader\; >bcode nick@​fruitbat [bleadperl]$ perl -MO=Bytecode\,-O2\,-Pmain\,-Psubs -e 'use subs "foo";

foo(); sub foo { print "bar\n"; }' >>bcode -e syntax OK nick@​fruitbat [bleadperl]$ perl bcode Undefined subroutine &main​::foo called at -e line 2.

Sadly not. Where do we go from here? Fiddle use subs; to generate different flags in the parent namespace?

Nicholas Clark

I'm not entirely sure where to go. One option (read​: (possibly evil) hack) is to have subs.pm store a list (perhaps @​B​::Bytecode​::Savelist) of what it's creating and have B​::Bytecode look at that to see what subs that it would normally ignore should actually should be saved. (Actually\, rather than checking for each sub\, it would probably just run over the list at the end\, since it already knows what it has already saved. The logic could be more or less the same as the function that saves the various call queues.) This could also be used more generally to allow programs to specifically only have certain subroutines saved (like a program that just needs File​::Spec​::catdir). Hmmm.

What say ye all?

-- BKS

__________________________________________________ Do You Yahoo!? Yahoo! Mail - Free email you can access from anywhere! http​://mail.yahoo.com/

[p5pRT]

From @nwc10

On Tue\, Aug 29\, 2000 at 07​:18​:40AM -0700\, Benjamin Stuhl wrote​:

I'm not entirely sure where to go. One option (read​: (possibly evil) hack) is to have subs.pm store a list (perhaps @​B​::Bytecode​::Savelist) of what it's creating and have B​::Bytecode look at that to see what subs that it would normally ignore should actually should be saved.

This seems both evil and slow :-( My understanding is that Evil is allowed if it has the saving grace

(Actually\, rather than checking for each sub\, it would probably just run over the list at the end\, since it already knows what it has already saved. The logic could be more or less the same as the function that saves the various call queues.) This could also be used more generally to allow programs to specifically only have certain subroutines saved (like a program that just needs File​::Spec​::catdir). Hmmm.

What say ye all?

If I change the body of subs.pm to

sub import {   my $callpack = caller;   my $pack = shift;   my @​imports = @​_;   foreach $sym (@​imports) {   eval "package ${callpack}; sub $sym;";   } };

then it works. However\, this is an eval "" so is slow. I don't know if/how it can done with (soft)references. (which also don't go pear shaped for oddly names values of $sym)

Alternatively\, the hack would be to make B​::Bytecode (or B; if this also affects the C compiler- i wasn't having any luck with making that work) implement package subs; with a slow version. I think that this would work because being -M on the command line B​::Bytecode will be pulled in before the compiled script\, so if it attacks %INC to stop the real subs.pm being used\, the script will not realise that use subs; is being "overloaded". And neither the real use subs; nor the script need to be nobbled.

But it would be nicer to find an efficient form of use subs; compatible with the Bytecode compiler to avoid this action at a distance.

Nicholas Clark

[p5pRT]

From @nwc10

On Tue\, Aug 29\, 2000 at 04​:57​:46PM +0100\, Nicholas Clark wrote​:

On Tue\, Aug 29\, 2000 at 07​:18​:40AM -0700\, Benjamin Stuhl wrote​:

I'm not entirely sure where to go. One option (read​: (possibly evil) hack) is to have subs.pm store a list (perhaps @​B​::Bytecode​::Savelist) of what it's creating and have B​::Bytecode look at that to see what subs that it would normally ignore should actually should be saved.

This seems both evil and slow :-( My understanding is that Evil is allowed if it has the saving grace

Simon Cozens and I had a chat at YAPC​::Europe\, and concluded that the correct solution was to turn off the GvIMPORTED_CV flag on a gv when the subroutine was defined. Hopefully this patch is the correct implementation of this.]

This works for the common case of​:

use subs qw(foo); foo; sub foo {   print "This is foo\n"; }

(which didn't before)

However\,

&bar; sub bar {   print "This is bar\n"; } use subs qw(bar);

still doesn't work. I guess it still needs sv.c not to set the GvIMPORTED_CV flag if you're importing from yourself to yourself. (simply to propagate any flag you find)

However\, with the patch one can now byte-compile the use subs; in perlcc

Nicholas Clark

Inline Patch

```diff --- op.c.orig Thu Sep 14 15:44:20 2000 +++ op.c Wed Sep 27 21:51:04 2000 @@ -4561,6 +4561,9 @@ CvGV(cv) = (GV*)SvREFCNT_inc(gv); CvFILE(cv) = CopFILE(PL_curcop); CvSTASH(cv) = PL_curstash; + /* use subs; etc may have turned this on + As we're (re)defining the sub here it's definately not imported */ + GvIMPORTED_CV_off(gv); #ifdef USE_THREADS CvOWNER(cv) = 0; if (!CvMUTEXP(cv)) { ```