Open AnibalMG opened 5 years ago
So the problem is basically that the packaging logic expects to find a module file Java/1.8.lua
, but there is none?
This is a tricky one to solve, since the .modulerc.lua
is not owned only by Java/1.8
, you could have another module wrapper for Java/1.9
as well for example (which would also be registered in .modulerc.lua
)...
Embed some sort of install-script that performs this step (in an otherwise empty RPM)?
It may make sense to consult @rtmclay on this one...
How does TACC handle installations that involve touching Java/.modulerc.lua
to register a short-hand version like 1.8
for 1.8.0_212
?
Well, we only install java occasionally and only the system one. But we do have a similar problem with our .version files for all packages.
What we do is have each rpm package that has a module file also create a .version.X.Y.Z file where X.Y.Z is the version (so a .version.1.8.0_212 in this case). Then a separate tool creates a symlink between .version and .version.X.Y.Z.
Normally, we can use the highest version to be the default but sometimes version 1.8 is preferred over 1.9 say. So in those cases, we create a symlink between the .version.X.Y.Z and .version
@rtmclay That approach won't work here though, since the Java/.modulerc.lua
may have to contain multiple module_version
statements, each of which tied to a different software version (and hence separate package)...
Can we avoid that somehow?
You can't. You can't have one file shared by multiple rpm. The solution is to not put .modulerc.lua in the rpm.
@rtmclay Is there a way to have multiple .modulerc
files in a single directory?
For the Java/1.8
and Java/11
wrappers, literally the only thing to install/change is the .modulerc.lua
, so not including it in the RPM is not really a good option ;)
Idea pitched by @Micket: how about letting Lmod support a .modulerc/
directory?
That way each Java
wrapper could own a separate file in that directory...
I don't understand what you are trying to do. Say we are creating an rpm for Java 1.8 we would create a file that we placed in the rpm called:
.modulerc-1.8.lua
For the rpm for Java 1.9 we would create:
.modulerc-1.9.lua
Then we use a separate tool to make a symbolic link. Say both java 1.8 and java 1.9 are installed. Then we have a tool which creates a symbolic link between .modulerc.lua and .modulerc-1.8.lua.
What is wrong with that? You only get one default. I don't know what a .modulerc.d directory would mean. You couldn't call it .modulerc because that would interfere with the old stuff. Sure you could set aliases. But what if you tried to set a default in each. What takes priority?
@rtmclay This is not related to defining a default version for Java
.
We use the .modulerc
to define aliases or wrappers, here are the contents of Java/.modulerc.lua
when both the Java/1.8
and Java/11
wrappers are installed:
module_version("Java/11.0.2", "11")
module_version("Java/1.8.0_212", "1.8")
The suggestion is to add support to let Lmod (also) pick up on something like .modulerc.d/Java-1.8.lua
:
module_version("Java/1.8.0_212", "1.8")
and .modulerc.d/Java-11.lua
:
module_version("Java/11.0.2", "11")
That way, the files are separate and can be included in the respective RPMs without a problem.
The actual names of the files in .modulerc.d
are mostly irrelevant (unless perhaps w.r.t. order for letting one overrule another, but that's not relevant from the EasyBuild side of it).
So why make Lmod do the work?. RPM's support post install scripts. Have the post install script take all the files in .modulerc.d and combine them into .modulerc.lua with "cat".
I forgot to mention that you'll need to also create a cleanup script which rebuilds the .modulerc.lua file after the old rpm is removed.
@rtmclay Well, ok, but that only works when EB's packaging feature is being used to create packages for software installations...
If EasyBuild is used to directly install software, then we'd need to keep doing what we do now (generating .modulerc.lua
directly). Or I guess in that case eb
could take the entries from .modulerc.d
and combine them into .modulerc.lua
, while taking some care not to kick out any entries from .modulerc.lua
...
Anyway, I need to think about this some more, your suggestion could work, but makes things quite complicated imho...
Hello, guys! I've made some changes to avoid this kind of issue in my fork. Can you look at it and review it? It's maybe a potential solution for such cases https://github.com/Mycron/easybuild-framework/blob/85bd95d974e59fb87e381ea71cd85c0b46201ec5/easybuild/tools/package/utilities.py#L139
@Mycron Great! It works for me!
Just a thought, Why we don't create a Lua module for Java-11
instead of modifying the modulerc
file?
Very late reply, but since this issue isn't resolved, i'll just shed some light on the issue:
A Java/11
module would conflict with the actual versions e.g. Java/11.x.x.x
.
Unfortunately, the workaround @Mycron made also completely breaks if you
Java-11.eb
+ Java-13.eb
as both would try to modify the same fileCurrently I'm thinking, assuming a modern Lmod is in use, one can just have the RPM completely empty. Shortened module names like "Python/3" already works.
Example, with .modulerc:
$ module load Groovy/2.5.9-Java-11
$ module list
Currently Loaded Modules:
1) Java/11 -> Java/11.0.20 2) Groovy/2.5.9-Java-11
without any .modulerc:
$ module purge
$ mv .modulerc.lua .modulerc.lua_bak
$ module load Groovy/2.5.9-Java-11
$ module list
Currently Loaded Modules:
1) Java/11.0.20 2) Groovy/2.5.9-Java-11
It will still work for these simple cases for anything that uses depends_on("Java/11")
For cases like STAR-CCM+ where we have
module_version("STAR-CCM+/18.06.006-r8", "2310-r8")
it's trickier. Making the fpm add a post script that modified the .modulerc should be possible though. But, if we just want to support Java and some simpler cases like that, then just making the java wrapper libraries do literally nothing will solve this issue.
I am getting an error when I try to generate the rpm for Java-1.8.eb. FPM could not fine the lua files because the modulerc easyblock is only genereting the .modulerc file.
I saw the issue on EB 3.7.1 and 3.8.0.