Perl / perl5

🐪 The Perl programming language
https://dev.perl.org/perl5/
Other
1.98k stars 560 forks source link

[feature] Configure option to add paths to INC without the funny business #17877

Closed kentfredric closed 3 years ago

kentfredric commented 4 years ago

There's presently only two options for adding paths to @INC, and both of them are unsuitable for my use.

Keep in mind, this is in a context where I'm already well aware of how ABI works, and I'm quite capable of deciding what is, and what isn't, a valid directory to put in @INC myself.

Current Option -Dinc_version_list

This is useless, because it takes what you pass it as a suggestion, and then internally mangles it, giving a different output than was asked for.

On one build, I passed:

-Dinc_version_list=5.30.3/x86_64-linux 5.30.3 5.30.2/x86_64-linux 5.30.2 5.30.0/x86_64-linux 5.30.0 5.24.0 5.22.2 5.16.0 5.12.10 5.12.4 5.8.0 

With the intent on building a perl with its INSTLIB being "/usr/lib64/perl5/5.30" (or something like that, not the intentionally missing trailing digit ), and something "clever" in Perl gives me this:

/etc/perl
/usr/local/lib64/perl5/5.30/x86_64-linux
/usr/local/lib64/perl5/5.30
/usr/lib64/perl5/vendor_perl/5.30/x86_64-linux
/usr/lib64/perl5/vendor_perl/5.30
/usr/local/lib64/perl5/5.30
/usr/lib64/perl5/vendor_perl/5.30
/usr/lib64/perl5/5.30/x86_64-linux
/usr/lib64/perl5/5.30

And importantly, this strategy seems impossible to use to build perl's for to explicitly include directories that dont exist on your system, because you know they do exist on the deployment system(s).

But the TL;DR version is "Perl is trying to be too clever", I could unpick that clever and work out where its going wrong, but I don't have the patience to bash my head into that wall, I just need something that listens to what I say and does it exactly how I said to do it, regardless of how foolish that may be.

Current option -Dotherlibdirs

This, is differently useless, in that although it does allow you to build with support for directories that don't exist, ... it then gets extra clever and decides to tack on archlib dirs to it.

Which is expressly something I dont want to do, and didn't ask for.

The objective here is to use these dirs as a "last resort backup plan", so that during upgrades, all pureperl packages that have yet to be reinstalled into the "new perl's" INSTDIR continue to work as best as possible (because sometimes, the package manager gets build order wrong and tries to use things that rely on things that are effectively broken by the upgrade of perl, due to falling outside of @INC, and as much as I try, there is nothing I can do to make that problem go away, other than to make perl less likely to break in that situation ). And yes, that could break when there are XS deps in the loop, and that's an understood caveat. But it would be better to not have those XS deps in the loadable path, and it would be better for perl to croak with a "can't find XS module foo" ( which may even safe-fail inside an eval ), than to have perl try to load a binary object with incompatible ABI.

My bodgearound

So, what I've hacked together instead, is this terrible bodge, because I don't fully understand the mess of C and Configure involved. But it works!

From 7572be1afb16621a537e9da02f96cf04cb10b655 Mon Sep 17 00:00:00 2001
From: Kent Fredric <kentnl@gentoo.org>
Date: Sat, 20 Jun 2020 22:14:22 +1200
Subject: Add support for -Dgentoolibdirs

Which just adds the libdirs verbatim in the right place without
having perl molest it in the process.
---
 Configure        | 26 ++++++++++++++++++++++++++
 config_h.SH      |  5 +++++
 perl.c           |  1 +
 perl_inc_macro.h |  8 ++++++++
 uconfig.sh       |  1 +
 5 files changed, 41 insertions(+)

diff --git a/Configure b/Configure
index cef8c3c078..77258e8f43 100755
--- a/Configure
+++ b/Configure
@@ -1181,6 +1181,8 @@ orderlib=''
 ranlib=''
 d_perl_otherlibdirs=''
 otherlibdirs=''
+gentoolibdirs=''
+d_gentoolibdirs=''
 package=''
 spackage=''
 pager=''
@@ -8101,6 +8103,28 @@ esac
 set d_perl_otherlibdirs
 eval $setvar

+case "$gentoolibdirs" in
+''|' ') dflt='none' ;;
+*) dflt="$gentoolibdirs" ;;
+esac
+$cat <<EOM
+Enter a colon-seperated list of explicit gentoo paths to stuff in @INC
+unmolested, or enter 'none' for no extra paths
+
+EOM
+rp='Colon-seperated list of gentoo-specific perl library search dirs?'
+. ./myread
+case "$ans" in
+' '|''|none) gentoolibdirs=' ';;
+*) gentoolibdirs="$ans" ;;
+esac
+case "$gentoolibdirs" in
+' ') val=$undef ;;
+*) val=$define ;;
+esac
+set d_gentoolibdirs
+eval $setvar
+
 : DTrace support
 dflt_dtrace='/usr/sbin/dtrace'
 $test -x /usr/bin/dtrace && dflt_dtrace='/usr/bin/dtrace'
@@ -24402,6 +24426,7 @@ d_openat='$d_openat'
 d_pathconf='$d_pathconf'
 d_pause='$d_pause'
 d_perl_otherlibdirs='$d_perl_otherlibdirs'
+d_gentoolibdirs='$d_gentoolibdirs'
 d_phostname='$d_phostname'
 d_pipe2='$d_pipe2'
 d_pipe='$d_pipe'
@@ -24937,6 +24962,7 @@ orderlib='$orderlib'
 osname='$osname'
 osvers='$osvers'
 otherlibdirs='$otherlibdirs'
+gentoolibdirs='$gentoolibdirs'
 package='$package'
 pager='$pager'
 passcat='$passcat'
diff --git a/config_h.SH b/config_h.SH
index 08c5923ef8..5563082969 100755
--- a/config_h.SH
+++ b/config_h.SH
@@ -1308,6 +1308,11 @@ sed <<!GROK!THIS! >$CONFIG_H -e 's!^#undef\(.*/\)\*!/\*#define\1 \*!' -e 's!^#un
  */
 #$d_perl_otherlibdirs PERL_OTHERLIBDIRS "$otherlibdirs"        /**/

+/* GENTOO_LIBDIRS:
+ * Like PERL_OTHERLIBDIRS, but doesn't stuff ARCH dirs in when not wanted
+ */
+#$d_gentoolibdirs GENTOO_LIBDIRS "$gentoolibdirs"      /**/
+
 /* PRIVLIB:
  * This symbol contains the name of the private library for this package.
  * The library is private in the sense that it needn't be in anyone's
diff --git a/perl.c b/perl.c
index 2013a76026..62e67cfaa2 100644
--- a/perl.c
+++ b/perl.c
@@ -4750,6 +4750,7 @@ S_init_perllib(pTHX)
     INCPUSH_ARCHLIB_EXP
     INCPUSH_PRIVLIB_EXP
     INCPUSH_PERL_OTHERLIBDIRS
+    INCPUSH_GENTOO_LIBDIRS
     INCPUSH_PERL5LIB
     INCPUSH_APPLLIB_OLD_EXP
     INCPUSH_SITELIB_STEM
diff --git a/perl_inc_macro.h b/perl_inc_macro.h
index 5a2f20dfae..4b69b39199 100644
--- a/perl_inc_macro.h
+++ b/perl_inc_macro.h
@@ -143,6 +143,14 @@
              INCPUSH_ADD_OLD_VERS|INCPUSH_CAN_RELOCATE);
 #endif

+#ifdef GENTOO_LIBDIRS
+# define INCPUSH_GENTOO_LIBDIRS S_incpush_use_sep(aTHX_ STR_WITH_LEN(GENTOO_LIBDIRS), \
+    INCPUSH_ADD_OLD_VERS|INCPUSH_CAN_RELOCATE);
+#endif
+#ifndef INCPUSH_GENTOO_LIBDIRS
+# define INCPUSH_GENTOO_LIBDIRS
+#endif
+
 #ifdef PERL_OTHERLIBDIRS
 #  define INCPUSH_PERL_OTHERLIBDIRS_ARCHONLY  S_incpush_use_sep(aTHX_ STR_WITH_LEN(PERL_OTHERLIBDIRS), \
              INCPUSH_ADD_OLD_VERS|INCPUSH_ADD_ARCHONLY_SUB_DIRS|INCPUSH_CAN_RELOCATE);
diff --git a/uconfig.sh b/uconfig.sh
index 1d4a0f4a57..fba825acad 100644
--- a/uconfig.sh
+++ b/uconfig.sh
@@ -185,6 +185,7 @@ d_futimes='undef'
 d_gai_strerror='undef'
 d_gdbm_ndbm_h_uses_prototypes='undef'
 d_gdbmndbm_h_uses_prototypes='undef'
+d_gentoolibdirs='undef'
 d_getaddrinfo='undef'
 d_getcwd='undef'
 d_getespwnam='undef'
-- 
2.27.0

The important part being INCPUSH_ADD_OLD_VERS|INCPUSH_CAN_RELOCATE without ADD_ARCHONLY_SUBDIRS

And the end result?

        /etc/perl
        /usr/local/lib64/perl5/5.30/x86_64-linux
        /usr/local/lib64/perl5/5.30
        /usr/lib64/perl5/vendor_perl/5.30/x86_64-linux
        /usr/lib64/perl5/vendor_perl/5.30
        /usr/lib64/perl5/5.30/x86_64-linux
        /usr/lib64/perl5/5.30
        /usr/lib64/perl5/vendor_perl/5.30.3/x86_64-linux
        /usr/lib64/perl5/vendor_perl/5.30.3
        /usr/lib64/perl5/vendor_perl/5.30.2/x86_64-linux
        /usr/lib64/perl5/vendor_perl/5.30.2
        /usr/lib64/perl5/vendor_perl/5.30.0/x86_64-linux
        /usr/lib64/perl5/vendor_perl/5.30.0
        /usr/lib64/perl5/5.24.0
        /usr/lib64/perl5/5.16.0
        /usr/lib64/perl5/5.12.4

And this is superior in every way to the options presently made available by configure.

Footnote

Yes, some of this drama precipitates due to having 2 different version schemes in employ for the layout. Full version 5.x.y is the standard pattern, and 5.x is something I'm simply experimenting with, because other problems have reared there head lately due to the aforementioned "oh noes, I can't specify directories that don't exist and have them work" issue.

eg:

  1. Bob builds a binary package of perl 5.30.1 and publishes it
  2. Bob builds a binary package of Foo-1.0 against perl 5.30.1 and publishes it, (embedding the 5.30.1 specific path in the package ).
  3. Alice builds a binary package of perl 5.30.2, on a system without any hint of 5.30.1 and publishes it.
  4. Mary installs Alices copy of Perl 5.30.2 from the binary package
  5. Mary installs Bobs binary package of Foo-1.0
  6. Foo-1.0 is now entirely unable to be used. EVEN THOUGH it is fully ABI inter-compatible with their current perl, due to the tiny quirk about @INC handling being terrible.

But at least in my case, one can be confident that we have the tools to automatedly determine the right paths, in advance, explicitly, and all that's needed is a good way to throw them over to perl without it getting "special" and undoing our work.

Perhaps something better than my crude hack can be employed.

kentfredric commented 4 years ago

Meant to say:

EVEN THOUGH it is fully ABI inter-compatible with their current perl, due to the tiny quirk about @INC handling being terrible.

And the code in question is PurePerl, does no funny business, and has existed unchanged for 20 years.

jkeenan commented 3 years ago

There's presently only two options for adding paths to @INC, and both of them are unsuitable for my use.

Keep in mind, this is in a context where I'm already well aware of how ABI works, and I'm quite capable of deciding what is, and what isn't, a valid directory to put in @INC myself.

[snip]

The OP is unfortunately no longer available to pursue his request in this ticket or to respond to questions about it. I've tried to wrap my head around it and can't claim to fully understand it. But I'm skeptical that we should consider adding Configure variables for particular OS distributions.

Thoughts from readers?

jkeenan commented 3 years ago

There's presently only two options for adding paths to @INC, and both of them are unsuitable for my use. Keep in mind, this is in a context where I'm already well aware of how ABI works, and I'm quite capable of deciding what is, and what isn't, a valid directory to put in @INC myself.

[snip]

The OP is unfortunately no longer available to pursue his request in this ticket or to respond to questions about it. I've tried to wrap my head around it and can't claim to fully understand it. But I'm skeptical that we should consider adding Configure variables for particular OS distributions.

Thoughts from readers?

No comments in nearly 8 months; closing.