search

easybuilders / easybuild-easyblocks

Collection of easyblocks that implement support for building and installing software with EasyBuild.
https://easybuild.io
GNU General Public License v2.0
106 stars 285 forks source link

also consider 'normalised' package name with underscore rather than dash in EasyBuild easyblock #3358

Closed boegel closed 5 months ago

boegel commented 5 months ago

Recent versions of setuptools (>=69.0.3?) enforce normalized package names by replacing dashes (-) with underscores (_) in the package name.

This affects the source tarballs of EasyBuild v4.9.2 (and likely also more recent versions going forward), so the custom easyblock for installing EasyBuild must consider package names with both dash (like easybuild-framework) and underscore (like easybuild_framework).

Without this, installing EasyBuild v4.9.2 with an existing EasyBuild release like v4.9.1 fails with an error like:

Failed to find required EasyBuild package easybuild-framework (subdirs: ['easybuild_easyconfigs-4.9.2', 'easybuild_framework-4.9.2', 'easybuild_easyblocks-4.9.2'], seldirs: [])")

That's because it's expecting to find a subdirectory like easybuild-framework-4.9.2 after unpacking the source tarball for easybuild-framework v4.9.2, but only easybuild_framework-4.9.2 is found (alongside the subdirectory for the unpacked easybuild-easyblocks and easybuild-easyconfigs packages).

This is annoying, since anyone who will be using eb --install-latest-eb-release or eb EasyBuild-4.9.2.eb (or equivalent, via --from-pr) will be hitting this, unless they use eb --include-easyblocks-from-pr 3358.

edit: more info on the reason why - is replaced with _ can be found in https://packaging.python.org/en/latest/specifications/binary-distribution-format/#escaping-and-unicode:

In distribution names, any run of -_. characters (HYPHEN-MINUS, LOW LINE and FULL STOP) should be replaced with _ (LOW LINE)

This is for names of binary distributions, for it also applies to source distributions, see https://packaging.python.org/en/latest/specifications/source-distribution-format/#source-distribution-file-name:

The file name must be in the form {name}-{version}.tar.gz, where {name} is normalised according to the same rules as for binary distributions
boegel commented 5 months ago

Test report by @boegel

Overview of tested easyconfigs (in order)

Build succeeded for 2 out of 2 (2 easyconfigs in total) node3111.skitty.os - Linux RHEL 8.8, x86_64, Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz, Python 3.6.8 See https://gist.github.com/boegel/2a8065571f3202fc47546cc5f144d2fd for a full test report.