OS Package dependencies use redhat syntax of -devel

[spikebike]

Debian and derivatives (like ubuntu) use -dev for developer packages.

So for instance OpenMPI-1.6.4-GCC-4.7.2.eb mentiones:

osdependencies = ['libibverbs-devel']

Which fails. Not sure if it would be better to have per OS naming, or just build as much as possible in easybuild.

The short term fix is to chance the name to -dev, which of course will break redhat based OS.

Hmm, I just search all the eb files and I only found 8 unique -devel packages references.

[ajdecon]

There's some ability to do OS detection in easyconfigs and specify dependencies correctly. As an example, I have PR #403 to do this for OpenMPI.

This is annoying, so I can see an argument for moving this into easybuild-framework and auto-renaming dependencies in the simple case (i.e. change all "-devel" dependencies into "-dev" if OS is Debian or Ubuntu). But I'm not sure it's robust, as it depends on the primary package name being the same on both distros. That's probably not going to be the case for all packages. Doing this per-easyconfig or per-easyblock would at least allow manually setting the names correctly for each distro.

[fgeorgatos]

Hi spikebike, ajdecon,

This issue relates also to (some discussed extensively):

• provide a more portable way of specifying os dependencies: https://github.com/hpcugent/easybuild-framework/issues/174
• allow escaping with --ignore-osdependencies: https://github.com/hpcugent/easybuild-framework/issues/102 In the latter, try to visit this url and its further pointers: http://enricozini.org/2011/debian/distromatch/

btw. Boost is a good (nasty ;-) case to check against in your explorations, because a) it is popular and many other software bits depends on it b) it can appear in many incarnations across distros (monolithic package or broken apart) c) it has its own mess over releases about what it supports and how For the above reasons, I think it's cool to have an easyblocofing about Boost ;-)

Yet, I'd rather prefer that things like libib* can be picked up correctly from upstream distro instead (very critical when the HPC folks that manage OS-level and user-level software stacks work apart from each other). In some cases, of externally supported systems, you really want to rely on the OS packages and not private builds.

[boegel]

@spikebike: This issue seems to be a duplicate of hpcugent/easybuild-framework#174, and is partially handled by #403.

Yes, we need a good long-term solution for this, and although I'm not a big fan of it, the approach take in #403 seems OK in the short term to resolve the issue package by package (I'm working with @ajdecon to get that merged in).

@spikebike: Are you OK with it if we close this one, and follow-up in hpcugent/easybuild-framework#174 for a suitable long-term solution?

[spikebike]

Well for a full environment the ./configures for each app has significant magic for detecting the presence of and version of includes and libraries for a build. Seems silly to replicate that inside easybuild. Granted it's manual, but the number of packages involved is minimal. In the short term I suggest using what was posted on a related ticket:

+# needed for --with-openib
+if OS_NAME in ['redhat', 'fedora', 'RHEL', 'SL', 'centos']:
+    osdependencies = ['libibverbs-devel']
+elif OS_NAME in ['debian', 'ubuntu']:
+    osdependencies = ['libibverbs-dev']

In the longer term I suggest a puppet like approach where you can just name a easybuild named package and there's a mapping of easybuild package names -> distro specific package names.

Currently said table would look something like for ubuntu: zlibdevel -> zlib1g-dev libibverbs-devel -> libibverbs-dev qt4-devel -> libqt4-dev libX11-devel -> libX11-dev

The nice thing is this is only for OS provided packages, and that really shouldn't be too much more than stuff necessary to build GCC (like a distro provided GCC), X11 (which seems pointless to build in EB), and whatever is specific to the kernel, like the infiniband/ofed packages.

Although now that I think about it. Maybe for 2.0 that EB should actually write the puppet module and the environment module. Then we could push whatever puppet is best at into puppet. Although that would be more ideal for network services like ganeti or hadoop instead of HPC oriented applications and libraries.

[boegel]

@spikebike: Great suggestion, I think providing a mapping in the framework is reasonable since the amount of entries should indeed be rather low.

I've added a note of your comment in hpcugent/easybuild-framework#174, which is the issue we'll be using for tracking this, and hence I'll close this issue...