ability to create meta-thingy (meta-rpm, meta-modulefile)

[sbinet]

it would be great to be able to create a modulefile (or a RPM) out of a group, depending on all the packages (and thus their associated modulefile or RPM) in that group.

• define a group:depends ?
• define a group-level set of features ?

(just bouncing some ideas)

[brettviren]

I'm not sure I follow.

You can set a configuration item at group level (or in [start]) and it will flow down to all packages in the group. You can use this (and I do) to set default group-level features. These, like all config items, may still be overridden by setting the item explicitly in the config section of some package in the group.

Also the dependencies expressed with the depends configuration item are honored by the modulefile feature and get translated into equivalent statements in the EM modulefile. These dependencies are expressed as individual packages and regardless of group. In principle one could express dependencies on an entire group (which I guess would still get translated into dependencies on individual packages in a modulesfile) but that would only save a bit of typing in the configuration file and it would overload the meaning of "group" which waf places.

Actually the group ordering and all the packages in a group are available to the features. They are attached to the bld object as orch_package_dict and orch_group_list. I don't think I'd feel comfortable overloading the interpretation of a "group" in the core of worch but doing something like this in an extra/external tool should not cause any problems that I can see.

So, a "fake" package could be added to the configuration which has a single feature, say called "modulegroup". This feature could then be written to produce a modulefile which merely expressed dependencies on the modulefiles of all the other packages in that group. In practice I haven't seen this need as so far all the suites I have been packaging have a single "main" application package that naturally depends on everything else. I guess this "modulegroup" would sort of produce something like a CMT "release" package. I could see this might be useful if you have some software not built by worch but which needs a group of packages which are.

Does this cover what you were going for or are you thinking about something else?

[sbinet]

yes, that's more or less what I was hinting at. see: https://github.com/atlas-org/lcg-builders/blob/master/build-lcg.cfg

the idea would be that by just running:

$ module load lcg-app

you'd get the whole (runtime) environment correctly configured.

more concretely, imagine you have the following groups:

[lcg-base]
 gcc
 mpfr, ppl, ...
 cmake,
 CMT

[lcg-externals]
 boost, python

you'd like to automatically have module load python correctly load all of the lcg-base runtime configs and have module load lcg-externals load boost, python and all of lcg-base.

creating a modulegroup feature seems relatively easy. generating a modulefile for each group, populated with a load package for each package in the group is indeed feasible thanks to the orch_group_list and orch_package_dict attributes. but, making sure the ordering of the groups in orch_group_list is the correct one is not obvious, at least with the current API. (well, I guess one could just ask people to list the groups in the correct order. that could just suffice as a first stab)

[brettviren]

Another approach is to invent a new configuration item, say modulegroup same as the feature name (or alternatively rpmgroup for some corresponding feature that produces RPMs) and apply it to each package in order to "tag" it as being in some group of that type. For example, maybe one has:

[package gcc]
modulegroup = lcg-base
features = ..., modulefile

# etc for mpfr, ppl, ...

[package lcg-base]
features = modulegroup
export_SOMEVAR = someval

Then the "modulegroup" feature of the lcg-base "fake" package can be used to walk all packages, find the ones with a matching modulegroup name and produce a lcg-base modulefile that mostly just depends on the modulefiles of the constituent packages but any other settings that might be defined can also be expressed (like the SOMEVAR in the example). In the case where waf groups coincide with modulegroups then the modulegroup configuration item, of course, only needs to be applied once in the appropriate [group] section.