Reduce Config bloat even with XS

[GoogleCodeExporter]

To re-produce this, I patched DynaLoader and Storable to not use Config. The 
patches can be provided here if you need them.

Then, I did this:
$>perl -e 'use Storable; Storable::fd_retrieve() if($ENV{NON_EXISTING_VAR}); 
print "$_\n"foreach (sort keys \%INC);'
Carp.pm
Exporter.pm
Exporter/Heavy.pm
Fcntl.pm
Storable.pm
XSLoader.pm
strict.pm
vars.pm
warnings.pm
warnings/register.pm

But when I compile it, I get this:
$>perlcc -O3 -r -e 'use Storable; Storable::fd_retrieve() 
if($ENV{NON_EXISTING_VAR}); print "$_\n"foreach (sort keys \%INC);'
Carp.pm
Config.pm
Config_git.pl
Config_heavy.pl
Errno.pm
Exporter.pm
Fcntl.pm
IO.pm
IO/File.pm
IO/Handle.pm
IO/Seekable.pm
SelectSaver.pm
Storable.pm
Symbol.pm
XSLoader.pm
warnings.pm

Original issue reported on code.google.com by todd.e.rinaldo on 13 Oct 2014 at 5:17

[GoogleCodeExporter]

My suspicion is that this is a result of bringing in DynaLoader? a simple hello 
world program doesn't seem to cause it.

Original comment by todd.e.rinaldo on 13 Oct 2014 at 5:19

[GoogleCodeExporter]

Original comment by todd.e.rinaldo on 13 Oct 2014 at 5:20

• Added labels: nick-issue

[GoogleCodeExporter]

To patch out use Config from Dynaloader, you can build perl by doing:

PERL_BUILD_EXPAND_CONFIG_VARS make all

But to be thorough, you also need to add this patch (at least to 5.14.4)

Original comment by todd.e.rinaldo on 13 Oct 2014 at 5:36

Attachments:

• Dynaloader.patch

[GoogleCodeExporter]

For Storable, I just patched sub CAN_FLOCK to always return 1 since on linux 
it's always true for us.

Original comment by todd.e.rinaldo on 13 Oct 2014 at 5:37

[GoogleCodeExporter]

Yes, Dynaloader and XSLoader < 5.15.3 require Config.

I only patched a special XSLoader::load_file which doesn't pull in Config, 
rather hardcodes the required Config values at compile-time. We need to patch 
DynaLoader, XSLoader and Storable to get rid of Config.
This new loader could be used with 5.14.4 also, I just never tested it.

Original comment by reini.urban on 13 Oct 2014 at 5:38

• Changed state: Accepted

[GoogleCodeExporter]

It will be pretty hard to get rid of Config.
You need to patch Storable, Errno, DynaLoader, XSLoader, 
and fix the compiler in 2 places:

diff --git lib/B/C.pm lib/B/C.pm
index a419777..defb0e6 100644
--- lib/B/C.pm
+++ lib/B/C.pm
@@ -4202,8 +4202,8 @@ sub B::GV::save {
        warn "GV::save \%$fullname\n" if $debug{gv};
        if ($fullname eq 'main::!') { # force loading Errno
          $init->add("/* \%! force saving of Errno */");
-         mark_package('Config', 1);  # Errno needs Config to set the EGV
-          walk_syms('Config');
+         #mark_package('Config', 1);  # Errno needs Config to set the EGV
+          #walk_syms('Config');
          mark_package('Errno', 1);   # B::C needs Errno but does not import $!
        } elsif ($fullname eq 'main::+' or $fullname eq 'main::-') {
          $init->add("/* \%$gvname force saving of Tie::Hash::NamedCapture */");
@@ -6363,7 +6363,7 @@ sub B::GV::savecv {
   return if $fullname eq 'B::walksymtable' or $fullname eq 'B::C::walksymtable';
   # Config is marked on any Config symbol. TIE and DESTROY are exceptions,
   # used by the compiler itself
-  if ($name eq 'Config') {
+  if ($fullname eq 'Config::Config') {
     mark_package('Config', 1) if !$include_package{'Config'};
   }
   $dumped_package{$package} = 1 if !exists $dumped_package{$package} and $package !~ /::$/;

But even then Config is still used somewhere.

Original comment by reini.urban on 13 Oct 2014 at 7:02

[GoogleCodeExporter]

FWIW:

--- a/lib/B/C.pm
+++ b/lib/B/C.pm
@@ -4147,14 +4147,10 @@ sub B::GV::save {
    warn "GV::save \%$fullname\n" if $debug{gv};
    if ($fullname eq 'main::!') { # force loading Errno
      $init->add("/* \%! force saving of Errno */");
-     mark_package('Config', 1);  # Errno needs Config to set the EGV
-          walk_syms('Config');
      mark_package('Errno', 1);   # B::C needs Errno but does not import $!
    } elsif ($fullname eq 'main::+' or $fullname eq 'main::-') {
      $init->add("/* \%$gvname force saving of Tie::Hash::NamedCapture */");
           if ($] >= 5.014) {
-            mark_package('Config', 1);  # DynaLoader needs Config to set the 
EGV
-            walk_syms('Config');
             svref_2object(\&{'Tie::Hash::NamedCapture::bootstrap'})->save;
           }
      mark_package('Tie::Hash::NamedCapture', 1);
@@ -6706,7 +6702,6 @@ sub save_unused_subs {
   }
   if ($use_xsloader) {
     force_saving_xsloader();
-    mark_package('Config', 1); # required by Dynaloader and special cased 
previously
   }
 }

@@ -6815,8 +6810,6 @@ sub save_context {
       use strict 'refs';
       if (!$include_package{'Errno'}) {
    $init->add("/* force saving of Errno */");
-   mark_package('Config', 1);
-        walk_syms('Config');
    mark_package('Errno', 1);
         svref_2object(\&{'Errno::bootstrap'})->save;
       } # else already included
-- 
2.1.2

Original comment by todd.e.rinaldo on 13 Oct 2014 at 8:51

[GoogleCodeExporter]

See the branch cpanel_build on github, which checks for env CPANEL_BUILD=1
and requires a patched Errno and DynaLoader. 
But perl5.14 still will pull in Config whenever a XS is used I fear. 

I'm also thinking of a new feature -fno-config which will replace all calls to 
Config::FETCH by the hardcoded compile-time Config value. Or as alias 
-fcompile-config

Original comment by reini.urban on 14 Oct 2014 at 7:47

• Changed title: Reduce Config bloat even with XS
• Changed state: Started

[GoogleCodeExporter]

Feel free to delete cpanel_build. We're maintaining those patches as they're 
specific to us.

Original comment by todd.e.rinaldo on 14 Oct 2014 at 5:21

[GoogleCodeExporter]

If we patch perl, the compiler needs to know that. And it might be beneficial 
to others to get a unbloated and fixed perl also. I keep track of compiler 
specific perl patches in Devel::PatchPerl::Plugin::Compiler

Original comment by reini.urban on 15 Oct 2014 at 7:27

[GoogleCodeExporter]

Original comment by reini.urban on 15 Oct 2014 at 7:29

• Added labels: Effort-Hard