perl: dynamic modules loading doesn't work

[Ri0n]

To compile my snmp software, I had to apply next patch to perl package

diff --git a/Makefile b/Makefile
index d3cb081d..ec5ca57b 100644
--- a/Makefile
+++ b/Makefile
@@ -77,18 +77,13 @@ endef

 # Static host perl
 define Host/Configure
-   ( cd $(HOST_BUILD_DIR); ./Configure -der -Uusedl -Duserelocatableinc -Dprefix=$(HOST_PERL_PREFIX) $(if $(CONFIG_PERL_THREADS),-Dusethreads,))
+   ( cd $(HOST_BUILD_DIR); ./Configure -der -Duserelocatableinc -Dprefix=$(HOST_PERL_PREFIX) $(if $(CONFIG_PERL_THREADS),-Dusethreads,))
 endef

 define Host/Install
    ( cd $(HOST_BUILD_DIR); ./miniperl installperl )
    $(INSTALL_DIR) $(HOST_PERL_PREFIX)/bin/
    $(CP) $(HOST_BUILD_DIR)/generate_uudmap $(HOST_PERL_PREFIX)/bin/
-   
-   # Link any possibly installed static extension in
-   $(MAKE) -C $(HOST_BUILD_DIR)/relink clean || true
-   ( cd $(HOST_BUILD_DIR)/relink && $(HOST_PERL_PREFIX)/bin/perl Makefile.PL )
-   $(call perlmod/host/relink,$(HOST_BUILD_DIR)/relink)
 endef

 # Target perl

I'm trying to understand is it something wrong with the perl package or with my software.

[Ri0n]

The mentioned software uses mib compiler which requires SNMP perl module, which can't be loaded w/o the patch

[symgryph]

I have this problem too, Where did you patch? I need to get dynamic perl modules working as well.

[kofec]

On Kirkwood platform: running "/mnt/openwrt/OpenwrtNSA310/staging_dir/hostpkg/usr/bin/perl" -I../../lib ppport_h.PL ppport.h in cpan/DB_File is up-to-date ppport.h in cpan/IPC-SysV is up-to-date ppport.h in cpan/Scalar-List-Utils is up-to-date ppport.h in cpan/Win32API-File is up-to-date ppport.h in dist/IO is up-to-date ppport.h in dist/PathTools is up-to-date ppport.h in dist/Time-HiRes is up-to-date removing temporary file PPPort.pm removing temporary file ppport.h LD_LIBRARY_PATH=/mnt/openwrt/OpenwrtNSA310/build_dir/target-arm_xscale_musl_eabi/perl/perl-5.28.1 ./miniperl -Ilib -I. make_ext.pl lib/auto/B/B.so MAKE="make" LIBPERL_A=libperl.so LINKTYPE=dynamic make[4]: Entering directory '/mnt/openwrt/OpenwrtNSA310/build_dir/target-arm_xscale_musl_eabi/perl/perl-5.28.1/ext/B' /mnt/openwrt/OpenwrtNSA310/build_dir/target-arm_xscale_musl_eabi/perl/perl-5.28.1/ext/B/../../../../../../staging_dir/hostpkg/usr/bin/perl "-I../../lib" -MExtUtils::Command::MM -e 'cp_nonempty' -- B.bs ../../lib/auto/B/B.bs 644 make[4]: Leaving directory '/mnt/openwrt/OpenwrtNSA310/build_dir/target-arm_xscale_musl_eabi/perl/perl-5.28.1/ext/B' LD_LIBRARY_PATH=/mnt/openwrt/OpenwrtNSA310/build_dir/target-arm_xscale_musl_eabi/perl/perl-5.28.1 ./miniperl -Ilib -I. make_ext.pl lib/auto/Compress/Raw/Bzip2/Bzip2.so MAKE="make" LIBPERL_A=libperl.so LINKTYPE=dynamic Generating a Unix-style Makefile Writing Makefile for Compress::Raw::Bzip2 Attempt to reload B.pm aborted. Compilation failed in require at ../../lib/CPAN/Meta/Requirements.pm line 90. at ../../lib/ExtUtils/MM_Any.pm line 1225. Attempt to reload B.pm aborted. Compilation failed in require at ../../lib/CPAN/Meta/Requirements.pm line 90. Unsuccessful Makefile.PL(cpan/Compress-Raw-Bzip2): code=65280 at make_ext.pl line 518. makefile:579: recipe for target 'lib/auto/Compress/Raw/Bzip2/Bzip2.so' failed make[3]: *** [lib/auto/Compress/Raw/Bzip2/Bzip2.so] Error 2 make[3]: Leaving directory '/mnt/openwrt/OpenwrtNSA310/build_dir/target-arm_xscale_musl_eabi/perl/perl-5.28.1' Makefile:138: recipe for target '/mnt/openwrt/OpenwrtNSA310/build_dir/target-arm_xscale_musl_eabi/perl/perl-5.28.1/.built' failed make[2]: *** [/mnt/openwrt/OpenwrtNSA310/build_dir/target-arm_xscale_musl_eabi/perl/perl-5.28.1/.built] Error 2 make[2]: Leaving directory '/mnt/openwrt/OpenwrtNSA310/feeds/packages/lang/perl' time: package/feeds/packages/perl/compile#6.51#0.84#7.30 package/Makefile:107: recipe for target 'package/feeds/packages/perl/compile' failed make[1]: *** [package/feeds/packages/perl/compile] Error 2 make[1]: Leaving directory '/mnt/openwrt/OpenwrtNSA310' /mnt/openwrt/OpenwrtNSA310/include/toplevel.mk:216: recipe for target 'package/perl/compile' failed make: *** [package/perl/compile] Error 2

[kofec]

And

running "/mnt/openwrt/OpenwrtNSA310/staging_dir/hostpkg/usr/bin/perl" -I../../lib ppport_h.PL
ppport.h in cpan/DB_File is up-to-date
ppport.h in cpan/IPC-SysV is up-to-date
ppport.h in cpan/Scalar-List-Utils is up-to-date
ppport.h in cpan/Win32API-File is up-to-date
ppport.h in dist/IO is up-to-date
ppport.h in dist/PathTools is up-to-date
ppport.h in dist/Time-HiRes is up-to-date
removing temporary file PPPort.pm
removing temporary file ppport.h
LD_LIBRARY_PATH=/mnt/openwrt/OpenwrtNSA310/build_dir/target-arm_xscale_musl_eabi/perl/perl-5.28.1 ./miniperl -Ilib -I. make_ext.pl lib/auto/B/B.so  MAKE="make" LIBPERL_A=libperl.so LINKTYPE=dynamic
make[5]: Entering directory '/mnt/openwrt/OpenwrtNSA310/build_dir/target-arm_xscale_musl_eabi/perl/perl-5.28.1/ext/B'
/mnt/openwrt/OpenwrtNSA310/build_dir/target-arm_xscale_musl_eabi/perl/perl-5.28.1/ext/B/../../../../../../staging_dir/hostpkg/usr/bin/perl "-I../../lib" -MExtUtils::Command::MM -e 'cp_nonempty' -- B.bs ../../lib/auto/B/B.bs 644
make[5]: Leaving directory '/mnt/openwrt/OpenwrtNSA310/build_dir/target-arm_xscale_musl_eabi/perl/perl-5.28.1/ext/B'
LD_LIBRARY_PATH=/mnt/openwrt/OpenwrtNSA310/build_dir/target-arm_xscale_musl_eabi/perl/perl-5.28.1 ./miniperl -Ilib -I. make_ext.pl lib/auto/Compress/Raw/Bzip2/Bzip2.so  MAKE="make" LIBPERL_A=libperl.so LINKTYPE=dynamic
make[5]: Entering directory '/mnt/openwrt/OpenwrtNSA310/build_dir/target-arm_xscale_musl_eabi/perl/perl-5.28.1/cpan/Compress-Raw-Bzip2'
/mnt/openwrt/OpenwrtNSA310/build_dir/target-arm_xscale_musl_eabi/perl/perl-5.28.1/cpan/Compress-Raw-Bzip2/../../../../../../staging_dir/hostpkg/usr/bin/perl "-I../../lib" -MExtUtils::Command::MM -e 'cp_nonempty' -- Bzip2.bs ../../lib/auto/Compress/Raw/Bzip2/Bzip2.bs 644
make[5]: Leaving directory '/mnt/openwrt/OpenwrtNSA310/build_dir/target-arm_xscale_musl_eabi/perl/perl-5.28.1/cpan/Compress-Raw-Bzip2'
LD_LIBRARY_PATH=/mnt/openwrt/OpenwrtNSA310/build_dir/target-arm_xscale_musl_eabi/perl/perl-5.28.1 ./miniperl -Ilib -I. make_ext.pl lib/auto/Compress/Raw/Zlib/Zlib.so  MAKE="make" LIBPERL_A=libperl.so LINKTYPE=dynamic
Parsing config.in...
Building Zlib enabled
Auto Detect Gzip OS Code..
Setting Gzip OS Code to 3 [Unix/Default]
Looks Good.
Generating a Unix-style Makefile
Writing Makefile for Compress::Raw::Zlib
Attempt to reload B.pm aborted.
Compilation failed in require at ../../lib/CPAN/Meta/Requirements.pm line 90.
 at ../../lib/ExtUtils/MM_Any.pm line 1225.
Attempt to reload B.pm aborted.
Compilation failed in require at ../../lib/CPAN/Meta/Requirements.pm line 90.
Unsuccessful Makefile.PL(cpan/Compress-Raw-Zlib): code=65280 at make_ext.pl line 518.
makefile:579: recipe for target 'lib/auto/Compress/Raw/Zlib/Zlib.so' failed
make[4]: *** [lib/auto/Compress/Raw/Zlib/Zlib.so] Error 2
make[4]: Leaving directory '/mnt/openwrt/OpenwrtNSA310/build_dir/target-arm_xscale_musl_eabi/perl/perl-5.28.1'
Makefile:138: recipe for target '/mnt/openwrt/OpenwrtNSA310/build_dir/target-arm_xscale_musl_eabi/perl/perl-5.28.1/.built' failed
make[3]: *** [/mnt/openwrt/OpenwrtNSA310/build_dir/target-arm_xscale_musl_eabi/perl/perl-5.28.1/.built] Error 2
make[3]: Leaving directory '/mnt/openwrt/OpenwrtNSA310/feeds/packages/lang/perl'
time: package/feeds/packages/perl/compile#6.71#0.86#7.52
package/Makefile:107: recipe for target 'package/feeds/packages/perl/compile' failed
make[2]: *** [package/feeds/packages/perl/compile] Error 2
make[2]: Leaving directory '/mnt/openwrt/OpenwrtNSA310'
package/Makefile:103: recipe for target '/mnt/openwrt/OpenwrtNSA310/staging_dir/target-arm_xscale_musl_eabi/stamp/.package_compile' failed
make[1]: *** [/mnt/openwrt/OpenwrtNSA310/staging_dir/target-arm_xscale_musl_eabi/stamp/.package_compile] Error 2
make[1]: Leaving directory '/mnt/openwrt/OpenwrtNSA310'
/mnt/openwrt/OpenwrtNSA310/include/toplevel.mk:216: recipe for target 'world' failed
make: *** [world] Error 2

kofec@E5420Mint /mnt/openwrt/OpenwrtNSA310 $ ls -al ./build_dir/hostpkg/perl/perl-5.28.1/lib/auto/Compress/Raw/Zlib/Zlib.so -rwxr-xr-x 1 kofec kofec 138648 Jun 12 13:49 ./build_dir/hostpkg/perl/perl-5.28.1/lib/auto/Compress/Raw/Zlib/Zlib.so

[rossbcan]

x86_64: exact same results as @kofec

• perl/host (dynamic patch per @Ri0n) compiles OK and ./build_dir/hostpkg/perl/perl-5.28.1/lib/auto/Compress/Raw/Zlib/Zlib.so exists
• perl/target compile fails same place (Attempt to reload B.pm aborted)

Any progress / workarounds on this?

[rossbcan]

solution here: https://forum.openwrt.org/t/perl-dynamic-modules/71746/2

[pprindeville]

@rossbcan Rather than a tarball, can you provide diffs as a PR?

[pprindeville]

@rossbcan Please respond.

[rossbcan]

@pprindeville

Waited 18 days? Shoulda nagged me sooner:)

PERL_VERSION:=5.28.1 Here's the patches: www.rossco.org/Downloads/Files/perl_dynamic_patches.tar.gz

I created the patches from my current build, just in case I fixed anything since first dealing with this issue. Tested x86_64, aarch64

Hope someone has more time than I to make perl dynamic the OpenWrt default.

...B

[pprindeville]

Waited 18 days? Shoulda nagged me sooner:)

If you want fast turnaround, learn to use GitHub properly and how to generate Pull Requests... this isn't a high bar.

[pprindeville]

@rossbcan I think your web server's PKI certificate is having some sort of issues...

[pprindeville]

Looking at the first patch, there are a lot of issues with it.

There are gratuitous whitespace changes, which are rejected by default.

As for:

 define Host/Configure
-       ( cd $(HOST_BUILD_DIR); ./Configure -der -Uusedl -Duserelocatableinc -Dprefix=$(HOST_PERL_PREFIX) $(if $(CONFIG_PERL_THREADS),-Dusethreads,))
+       #( cd $(HOST_BUILD_DIR); ./Configure -der -Duserelocatableinc -Dprefix=$(HOST_PERL_PREFIX) $(if $(CONFIG_PERL_THREADS),-Dusethreads,))
+       ( cd $(HOST_BUILD_DIR); ./Configure -der -Duserelocatableinc -Dprefix=$(HOST_PERL_PREFIX) $(if $(CONFIG_PERL_THREADS),-Dusethreads,))

No. Don't comment out a line then add it back with changes. The previous version gets recorded into git. That's what an SCM is for. Just have the new line there.

In:

+TARGET_CPPFLAGS += -L$(STAGING_DIR_ROOT)/lib -Wl,-rpath-link=$(STAGING_DIR_ROOT)/lib
+TARGET_CPPFLAGS += -L$(STAGING_DIR_ROOT)/usr/lib -Wl,-rpath-link=$(STAGING_DIR_ROOT)/usr/lib

Don't set it twice. Just use continuation lines. Like:

+TARGET_CPPFLAGS += -L$(STAGING_DIR_ROOT)/lib -Wl,-rpath-link=$(STAGING_DIR_ROOT)/lib \
+                   -L$(STAGING_DIR_ROOT)/usr/lib -Wl,-rpath-link=$(STAGING_DIR_ROOT)/usr/lib

Then there's this:

        mv $(PKG_BUILD_DIR)/Makefile $(PKG_BUILD_DIR)/Makefile_old
        cp ./files/Makefile $(PKG_BUILD_DIR)

No. You need a patch to Makefile.SH so that it generates the changes that you want.

Here:

+       sed -i 's#-I/usr/local/#-I$(STAGING_DIR_HOSTPKG)/include -I$(STAGING_DIR_HOST)/include -I$(STAGING_DIR_HOSTPKG)/usr/#g' $(HOST_BUILD_DIR)/config.sh
+       sed -i 's#-L/usr/local/#-L$(STAGING_DIR_HOSTPKG)/lib -L$(STAGING_DIR_HOST)/lib -L$(STAGING_DIR_HOSTPKG)/usr/#g' $(HOST_BUILD_DIR)/config.sh

Please don't run sed multiple times on the same file. -e with multiple commands is your friend.

For:

+# Variables for ./files/Makefile
+export TARGET_ARCH:=$(CONFIG_ARCH)
+export HOST_BUILD_DIR:=$(HOST_BUILD_DIR)
+export PKG_BUILD_DIR:=$(PKG_BUILD_DIR)
+export STAGING_DIR:=$(STAGING_DIR)
+export STAGING_DIR_ROOT:=$(STAGING_DIR_ROOT)
+export STAGING_DIR_HOSTPKG:=$(STAGING_DIR_HOSTPKG)
+export TOOLCHAIN_DIR:=$(TOOLCHAIN_DIR)
+## obsolete?  !Important (as root): ln -sf $(STAGING_DIR_HOSTPKG)/lib/libisc.so.1603.1.0 /usr/lib64/libisc.so.1603

if you're not changing the variable's value, you can just export the name without an assignment.

The 2nd patch I won't discuss because you're clobbering, not tweaking, a distributed (and synthesized) Makefile. Every time Perl gets updated, this will break badly. It's not maintainable.

Again, this would all be a lot easier to discuss with a proper PR.

[firmwarecostum]

is there a way to add libperl.so in the host? I noticed there are some files requiring this libperl.so in the host of one of the freeradius3 packages

[firmwarecostum]

@rossbcan it doesn't exist in openwrt gdbm/host bind/host db47/host