search

conda-forge / perl-feedstock

A conda-smithy repository for perl.
BSD 3-Clause "New" or "Revised" License
3 stars 32 forks source link

Use paths without minor and patch version for sitelib etc. #48

Open mbargull opened 3 years ago

mbargull commented 3 years ago

copied from https://github.com/conda-forge/conda-forge-pinning-feedstock/pull/936#issuecomment-725351898 :

Does anyone have capacity to look into the sitelib et al. configuration, re https://github.com/Perl/perl5/blob/v5.32.0/INSTALL#L593-L883 before we update this pin? If we'd use that for newer perl builds, we could do away with the PREFIX/lib/major.minor.patch directory which would let us avoid downstream rebuilds for every new patch release (assuming Perl packages don't hard-code the runtime version otherwise during build). IDK if we'd need to work around the prefix that contains the string "perl" thing from https://github.com/Perl/perl5/blob/v5.32.0/INSTALL#L826-L828 (because we always have .../conda-bld/perl... in our host prefix) or if setting the pahts (e.g., as they do at https://github.com/archlinux/svntogit-packages/blob/da42520d045c35a5db1425218539e82e300e9f39/trunk/PKGBUILD#L163-L173) suffices.

The prime thing would be to also build on Windows with the same paths (currently it seems to use Library/vendor/lib and Library/lib without a further subdirectory). We could then have noarch: generic packages also for Windows just like we have for R packages. I meant to look into Windows builds myself last winter/spring but didn't get to it. Revisiting this, I now only get exiftool for Perl libraries on Windows, at least with a quick grep for perl/Makefile.PL/skip on my local feedstocks copy. Which means it wouldn't have much impact anyway if we don't get a self-compiled Windows build soon.

mbargull commented 3 years ago

cc @conda-forge/bioconda-recipes

mbargull commented 3 years ago

NB: This issue has a high impact and possibly also urgency because

mbargull commented 3 years ago

Other random notes:

mingwandroid commented 3 years ago

I think you mean the all_replacements feature here @mbargull? Build id pat can be used to make SRC_DIR deterministic too in the first place though I suppose.

noarch generic would be cool too.

mbargull commented 3 years ago

I think you mean the all_replacements feature here @mbargull?

I meant a thing that would turn /opt/conda/conda-bld/perl_*/_h_env_* into /opt/conda/conda-bld/this-does-not-contain-p-e-r-l_*/_h_env_* -- but I might've missed/misremembered something; I only noticed it while skimming some patching-related PR of yours ;).

isuruf commented 3 years ago

How about perl extension modules? Are they compatible?

isuruf commented 3 years ago

Ping on this. This is blocking some packages

mbargull commented 3 years ago

I took a stab at this at gh-49. Only the non-Windows part -- I won't be able to look into Windows any time soon either...

How about perl extension modules? Are they compatible?

AFAIUI, those are the ones that go into the "archlib" directories, e.g., currently $PREFIX/lib/5.32.0/x86_64-linux-thread-multi, $PREFIX/lib/site_perl/5.32.0/x86_64-linux-thread-multi. With the layout from gh-49 they go into $PREFIX/lib/perl5/core_perl/5.32, $PREFIX/lib/perl5/site_perl/5.32 (as they should be compatible within the same minor Perl version, cf. https://github.com/Perl/perl5/blob/v5.32.0/INSTALL#L2444). By omitting the x86_64-linux-thread-multi part, we can have the same directory structure between different operating systems.

Non-compiled packages previously went into $PREFIX/lib/5.32.0, $PREFIX/lib/site_perl/5.32.0. In gh-49 they are at $PREFIX/lib/perl5/core_perl, $PREFIX/lib/perl5/site_perl and could be used for perl packages with different minor versions, cf. https://github.com/Perl/perl5/blob/v5.32.0/INSTALL#L97.

Overall, we can do away with the initial 5.32.0 directory and the x86_64-linux-thread-multi arch part because we don't support installing multiple Perl versions or different arches alongside each other.

gh-49 should lay the groundwork for future packaging schemes. If the review goes well and we merge it, we can relax our pinning from major.minor.patch to major.minor.* *_perl5 (on Linux/macOS at least). For future noarch: generic packages or generally packages without extension modules that don't need the minor version pinned, we may have to create some helper packages to ensure the correct pinnings and no files in the perl5/*/5.* paths. If someone wants to work on this, please do! I won't be able to get to this in the near future.

Also, there is this site and vendor split. Maybe we should recommend using the vendor target for our packages -- with perl Makefile.PL INSTALLDIRS=vendor or something a long those lines -- before someone starts to port over all those packages from Bioconda.

Ping on this. This is blocking some packages

Which ones? AFAIK we only have at most a couple of handful real Perl packages in conda-forge yet. I would guess that all/most that are blocked by this do not have to have perl pinned, i.e., a - perl * or - perl >=some-version would suffice, doesn't it?

isuruf commented 3 years ago

Which ones? AFAIK we only have at most a couple of handful real Perl packages in conda-forge yet. I would guess that all/most that are blocked by this do not have to have perl pinned, i.e., a - perl * or - perl >=some-version would suffice, doesn't it?

texinfo. I fixed a bunch of recipes to do that, but texinfo seems to be a perl extension module.

mbargull commented 3 years ago

I fixed a bunch of recipes to do that,

Nice!

but texinfo seems to be a perl extension module.

I've taken a look at it and built it with the artifacts from gh-49* and with the old 5.26.2. I'm not entirely sure what to make of it. It doesn't install it as Perl package under lib/perl5 (lib/5.26.2, lib/site_perl/5.26.2). It does record the Perl version in the lib/texinfo/*.so libs (as v5.32.0/v5.26.2). I don't know what that pinning that package expects (does it need 5.32, 5.32.0? or can it have relaxed >=5.32.0, *, ???).

*(i.e., changed the recipe pinning to perl: [5.32 *_perl5], pin_run_as_build: {perl: {max_pin: x.x}} plus just added it to requirements: build: because perl=5.26.2 was pulled in via autotools packages.)

mbargull commented 3 years ago

Oh, and to add {bin/*,lib/texinfo/*.so} don't link to anything Perl-specific, only to glibc (plus bin/info links to tinfo/ncurses).

Apparently it's "normal" to not link to libperl for extension modules. I built a few of our packages against the shared lib from gh-49 locally and checked Perl packages from my system distribution. The extension modules include (undefined) Perl_* symbols but do not link to libperl.

To sum it up: lib/texinfo/*.so do indeed seem to be "normal" Perl extension modules, just in an unusual location. Ergo, yes, @isuruf is right, this is blocking texinfo's osx-arm64 migration. (Plus, we built libperl.a before; gh-49 now also addresses that and builds libperl.so instead.)