Open judovana opened 3 months ago
To summarize, there are two requests for Adoptium to consider:
What does this actually mean?
For 1. Provide an RPM package that Provides specific versions of Java
RPMs have a dependency mechanism using strings via Provides, BuildRequires, and Requires fields in the RPM spec file. For example: https://github.com/adoptium/installer/blob/master/linux/jdk/redhat/src/main/packaging/temurin/21/temurin-21-jdk.spec#L77
For some historical reason, Temurin RPMs do not specify a version in the provide field.
Provides: java
If we did, it would so look like:
Provides: java = 1:21.0.2.0.0.13-2.fc40
In comparison, other vendors like Amazon, Azul, and Red Hat, have explicit versions for their RPM's Provides fields.
Why does this matter?
The lack of a version is problematic. If Temurin 21 is installed on the system, and the Fedora user installs a package that needs JDK 8, then it would be satisfied by Temurin 21, but not actually work.
For 2. Provide a headless RPM package.
This is a subpackage in the same spec file that has a related files and dependencies removed to run in a headless environment with a smaller footprint.
For example: Package defined in: https://src.fedoraproject.org/rpms/java-21-openjdk/blob/rawhide/f/java-21-openjdk.spec#_1413 Files list defined in: https://src.fedoraproject.org/rpms/java-21-openjdk/blob/rawhide/f/java-21-openjdk.spec#_2339 Which actually is defined in: https://src.fedoraproject.org/rpms/java-21-openjdk/blob/rawhide/f/java-21-openjdk.spec#_799
@judovana
Who will be using the headless packages? In the past we've had explicit requests for JRE packages, but I don't recall requests for headless packages. Are there many other packages in Fedora that depend on the headless package and can we see that list here? It would be nice to have some strong evidence for their benefit.
@judovana
You mentioned a meta-package solution for the problem of versionless provides. Can you elaborate on what that looks like?
And I still do not fully understand: Why can't we just change all the spec files to Provides like:
Provides: java = 1:21.0.2.0.0.13-2.fc40
What is the danger of such a change? I appreciate your expertise here.
I have a few questions regarding the proposal:
Proposed solution is to create a metapackage which will be providing the version-less provides. Such a package will not be shipped to repos offered to Fedora, but will be offered in current Temurin repos.
Just to be clear - are you suggesting we change the current package to be versioned-only and then the metapackage would depend on the versioned one and "provide" the versionless one? And
The rpms will replace themselves with repository file, and there will be proper obsolescence, so Temurin JDK(s) of proper versions will get transparently installed
Just to be 100% clear, does that mean that you'll replace the current java8,11,17 packages in Fedora with something that sets up the Temurin dnf repositories? And once that's done the user would be able to install temurin as a separate package? Or, since you've said it's transparent, would the Fedoras RPMs trigger the install as a postinstall step after setting up the temurin repos?
I will answer as soon as I can, bu those are not simple answer. One note on jdk8 and openjfx. In fedora we have opnejfx apckages whic are for jdk11 and up, those shoudl be worign wihtout issues. For jdk8, we ahd to add jdk8 subpackage, which added the necessary symlinks. This will disapear, but I think, we can already let openfx8 die in peace.
To summarize, there are two requests for Adoptium to consider:
1. Blocker: Provide an RPM package that Provides specific versions of Java 2. Regression: Provide a headless RPM package.
yes and yes. But introduction of headless subpackage is beneficial to all.
plus 3.regression is that javafx for jdk8, buthtat really do nto count at the end. plus 4.regression lsot of shenandoah for jdk8, I guess thats ok too.
What does this actually mean?
For 1. Provide an RPM package that Provides specific versions of Java
RPMs have a dependency mechanism using strings via Provides, BuildRequires, and Requires fields in the RPM spec file. For example: https://github.com/adoptium/installer/blob/master/linux/jdk/redhat/src/main/packaging/temurin/21/temurin-21-jdk.spec#L77
For some historical reason, Temurin RPMs do not specify a version in the provide field.
Provides: java
If we did, it would so look like:
Provides: java = 1:21.0.2.0.0.13-2.fc40
In comparison, other vendors like Amazon, Azul, and Red Hat, have explicit versions for their RPM's Provides fields.
Why does this matter?
I'm not sure what version you now refer. I have never ever mentioned version behind versionless provides. Whether you have
Provides: java
or
Provides: java = 1:21.0.2.0.0.13-2.fc40
Does not meter from my point of view. Still the second one is more correct, although we recenlty hit issue where versionless provides (eg java
should not have the version string = 1:21.0.2.0.0.13-2.fc40
. Unluckily this fix is not yetr in fedora, and in addtition our jira nd cgit is currently offline :-/ I need to double check this information before we progress.
Anyway the version-full , eg java-21-openjdk
, provides must remain fully versiond by version string of = ....
The lack of a version is problematic. If Temurin 21 is installed on the system, and the Fedora user installs a package that needs JDK 8, then it would be satisfied by Temurin 21, but not actually work.
I'm again not sure what you mean by lack of version. Sorry.
The whole versionless x versionfull issue is based on java
java-devel
javadoc
.... versus java-XY
java-xy-devel
javadoc-xy
... See usage of is_system_jdk variable.
If the jdk is system jdk (is_system_jdk 1) then a version-less provides like java
java-devel
... are provided. If not ((is_system_jdk 0) then only version-full like java-XY
java-xy-devel
provides are provided.
The whole point is to distinguish system jdk against any other jdk. Only system jdk have versionless provides. This allows us to move system jdk without touches to java aplications. This system have evolved in years, and is perfectly working.
You will hit the https://src.fedoraproject.org/rpms/java-21-openjdk/blob/rawhide/f/java-21-openjdk.spec#_393 and https://src.fedoraproject.org/rpms/java-21-openjdk/blob/rawhide/f/java-21-openjdk.spec#_921 and https://src.fedoraproject.org/rpms/java-21-openjdk/blob/rawhide/f/java-21-openjdk.spec#_1013 lines, which is making the issue a bit more tricky, but still proeprly doable.
If package requires some exact java, it requires it via java-xy...
provides. But it is allowed only for special cases.
That is reason, why temurin jdks, when used in fedora, must not contain the versionless provides. Then exactly what you describe The lack of a version is problematic. If Temurin 21 is in...
would happen.
Howeverm I think it is correct that each temurin jdk as it is now in generic repos, provides versionless provides like "java", as it is then trully versatile as any portbale package should be. If you decide to drop version-less provides completely, then we are done.
Pls consult also Deepak on this topic. it was moreover me an him behind this change, he may remember additional points
if not, a metapackage is here to help:
You mentioned a meta-package solution for the problem of versionless provides. Can you elaborate on what that looks like?
The metapackage is package with only with provides. Such package require the real package, which itself do not have the provides.
So in this exact scope it may be (but I hope it will be done better):
If fedora frienldy names are no go, thats ok, virtual provides (which you afaik already have) would do. Only the meta packages will be weirdly named, and some rename of original packages is necessary anyway.
As your rpsm are "only" repacking tarball, maybe better then mtapackage is simply generate special set of rpms jsut for fedora, without versionless provides. As second repo is in play anyway.
And I still do not fully understand: Why can't we just change all the spec files to Provides like: Provides: java = 1:21.0.2.0.0.13-2.fc40
Because it is not what I ment by versionless x versinfull ;) Unluckily simple additon of version behind java do not solve the system jdk problem. Pls consult with deepak if I'm not around, this is reallytricky issue "why we can not just add the version string behind versionless provides"
What is the danger of such a change? I appreciate your expertise here. Of added version string?
Unluckily simple additon of version behind java do not solve the system jdk problem.
Of meta package? No danger, only maintenance burden.For 2. Provide a headless RPM package.
This is a subpackage in the same spec file that has a related files and dependencies removed to run in a headless environment with a smaller footprint.
For example: Package defined in: https://src.fedoraproject.org/rpms/java-21-openjdk/blob/rawhide/f/java-21-openjdk.spec#_1413 Files list defined in: https://src.fedoraproject.org/rpms/java-21-openjdk/blob/rawhide/f/java-21-openjdk.spec#_2339 Which actually is defined in: https://src.fedoraproject.org/rpms/java-21-openjdk/blob/rawhide/f/java-21-openjdk.spec#_799
Right. better view on that is, what files are not in headless one: https://src.fedoraproject.org/rpms/java-21-openjdk/blob/rawhide/f/java-21-openjdk.spec#_791
Only rpm generated requires on those .so are half a gig at least. And thats answers also:
Who will be using the headless packages? In the past we've had explicit requests for JRE packages, but I don't recall requests for headless packages. Are there many other packages in Fedora that depend on the headless package and can we see that list here? It would be nice to have some strong evidence for their benefit.
The user is anybody who do not want to mess its system by X and other windowing depndecs, no metter what reason it is. Cool example is any build system. But unluckily java-devel rewuires full jre, not jsut jre-headless. java-headless is actually a default requires anybody should use, unless they really requires X for rutime. So the answer for you is every deployment and every cli user who do not need devel which is quite a lot.
I have a few questions regarding the proposal:
Proposed solution is to create a metapackage which will be providing the version-less provides. Such a package will not be shipped to repos offered to Fedora, but will be offered in current Temurin repos.
Just to be clear - are you suggesting we change the current package to be versioned-only and then the metapackage would depend on the versioned one and "provide" the versionless one? And
Correct. For details see my answer to Jie.
The rpms will replace themselves with repository file, and there will be proper obsolescence, so Temurin JDK(s) of proper versions will get transparently installed
Just to be 100% clear, does that mean that you'll replace the current java8,11,17 packages in Fedora with something that sets up the Temurin dnf repositories? And once that's done the user would be able to install temurin as a separate package? Or, since you've said it's transparent, would the Fedoras RPMs trigger the install as a postinstall step after setting up the temurin repos?
it can be both, and depends on our agreement.
Smallest intervention - I will just discontinue jdk8,11 and in a year also 17 and will enhance fedora java pages so they recommend to install temurin jdk. With link to temurin pages.
Bigger intervention - I will replace jdk 8,11 and later 17 really by the repository file, but will not transparently allow update to temurins. This and additional steps will be documented in fedora-java pages.
Total intervention. I will replace jdk 8,11 and later 17 really by the rrepository file, and will set up proper obsoleted
by in my rpms and/or proepr provides in temurins so it will allow transaprent update to temurins. I guess It will be also documented in fedora-java pages.
I see the last as best, but it is also on FESCO and fedora-devel discussion. And of course Temurin people .
If third will be the option, then I do not plan to do this in posttrans. I plan to add the repository in one transaction, and later to remove myslef and substitue by temurins.
I had jsut realised that jdk8 will lost also shenandoah, but I guess thats ok.
Is jdk11 built with shenandoah enabled? Upstreamis, so I guess it is.. Please correctme
Thank you Jiri for clarifying. Sadly the GH Comment UI is not that great for replying in-line as it gets longer as longer :( I wonder if there is some other tool to help with that...
For 1. Provide an RPM package that Provides specific versions of Java
Okay. I'm glad you've clarified further as I misunderstood the request. A concrete example would have been nice in the initial request.
So, what is actually requested:
Before:
Provides: java
...
After:
Provides: java-21
...
Before continuing further discussion, is this correct? I think so as now all your other comments make complete sense in this context :)
The term version-less provides was confusing. I thought you were referencing the = 1:21.0.2.0.0.13-2.fc40
portion of the Provides field as that is a version, and we do not have that, therefore it is version-less.
Thank you Jiri for clarifying. Sadly the GH Comment UI is not that great for replying in-line as it gets longer as longer :( I wonder if there is some other tool to help with that...
For 1. Provide an RPM package that Provides specific versions of Java
Okay. I'm glad you've clarified further as I misunderstood the request. A concrete example would have been nice in the initial request.
yah, reading it, now, with this ponit of view on, I should be more specific, but the issue is so obvious to me:( Really sorry.
So, what is actually requested:
Before:
Provides: java ...
After:
Provides: java-21 ...
Before continuing further discussion, is this correct? I think so as now all your other comments make complete sense in this context :)
yes.
It contains also various:
Provides: jre-21
Provides: jre-21-temurin
Provides: jre-21-openjdk
Provides: java-21
Provides: java-sdk-21-temurin
Provides: java-sdk-21-openjdk
Provides: java-sdk-21
Provides: java-21-devel
Provides: java-21-temurin-devel
Provides: java-21-openjdk-devel
...
I recommend, that all version-full provides contains full version string = %{epoch}:%{version}-%{release}
The term version-less provides was confusing. I thought you were referencing the
= 1:21.0.2.0.0.13-2.fc40
portion of the Provides field as that is a version, and we do not have that, therefore it is version-less.
Correct. In my vocabulaty thet = 1:...
is version string. Sorry for confusion. Am happy it was cleared at the end.
Does not meter from my point of view. Still the second one is more correct, although we recenlty hit issue where versionless provides (eg
java
should not have the version string= 1:21.0.2.0.0.13-2.fc40
. Unluckily this fix is not yetr in fedora, and in addtition our jira nd cgit is currently offline :-/ I need to double check this information before we progress.
here is the jira: https://issues.redhat.com/browse/OPENJDK-2507
The "provides" value has changed in java-1.8.0-openjdk-headless.x86_64,
version 1:1.8.0.392.b08-2.el7_9, rpm.
Previous versions of this package listed "java-headless = 1:1.8.0", and
u392 has a specific minor version tied to it: "java-headless =
1:1.8.0.392.b08-2.el7_9".
The result is that third-party packages that require specifically
"java-headless 1:1.8.0" are not compatible with the new version 392.
As this revert goes only to old rhels, not to the future, temurins should use full version string also for versionless provides.
Thanks @judovana for all your follow ups. The explanations make sense to me. Delivering headless subpackages seems generally worthy as it will strengthen our offerings. Our installers in general do need some love...
Smallest intervention - I will just discontinue jdk8,11 and in a year also 17 and will enhance fedora java pages so they recommend to install temurin jdk. With link to temurin pages.
Bigger intervention - I will replace jdk 8,11 and later 17 really by the repository file, but will not transparently allow update to temurins. This and additional steps will be documented in fedora-java pages.
Total intervention. I will replace jdk 8,11 and later 17 really by the rrepository file, and will set up proper obsoleted by in my rpms and/or proepr provides in temurins so it will allow transaprent update to temurins. I guess It will be also documented in fedora-java pages.
For the three levels of intervention, in the case of the smallest/bigger, is a fix for the Provides necessary? It sounds like it would be necessary for the total intervention case at least.
The Temurin RPM already clashes with the system JDK in Fedora today, right? So the motivation to fix it is that, after your change in Fedora, more users will be using Temurin on Fedora systems and the problem will likely be seen more often. Just checking, are there any other motivations for the change?
I think the removal of versionless provides is mandatory in all three cases (moreover as you summ up in last paragraph).
Moreover expected case:
Unexpected case:
Okay. Thanks for those two examples @judovana, that is really helpful to understand. So reviewing this in whole, I see two different parts.
The headless subpackage. I do not think this requires a full PMC review. It would be an additional feature that strengthens our installer offering and does not have any compatibility issues for existing users.
The versionless provides. The manner in which we implement this will impact existing users. We should have a PMC review before proceeding.
Lets continue to discuss versionless provides here and put the headless feature into it's own issue.
Note, that this issue should serve to find all possible issues in both Temurin JDK and in Fedora, so that the feature can happen at all.
If we agreed both headless subpackage and versionless provides should happen, then they both desrve its own issue, and we can continue here to search for other issue, or disucss them there as they will be occuring once it it made more public
Note, that this issue should serve to find all possible issues in both Temurin JDK and in Fedora, so that the feature can happen at all.
If we agreed both headless subpackage and versionless provides should happen, then they both desrve its own issue, and we can continue here to search for other issue, or disucss them there as they will be occuring once it it made more public
Okay, makes sense. Let's follow this approach then. Do you see any other areas to investigate for issues with Temurin and Fedora?
Nope, But I'm pretty sure they will emerge:(
Good to see the discussion taking place here. I'm going to remove the PMC tag for this specific issue as it is getting long, and ask that we open associated specific focused issues for decisions concluded here that need to go to PMC review. There is no rush for these.
This is quite long but I'm adding in this comment as a summary for my own benefit and for anyone else reading it as an explanation of the concern with what Temurin currently does that blocks Fedora using our rpm files. This is specific to the versioning and does NOT cover any discussion relating to headless.
Fedora is considering removing their older openjdk packages and encouraging users to move to Temurin instead. They have a number of options for this, however a blocker to any user migration story is that they need RPM packages that Provides specific versions of Java and do not Provide: java
which the current temurin ones do (See Temurin redhatspec file)
Fedora expects that any package which satisfies the requirement for java
will be the one that was typically the latest java LTS specification (Currently java 21) at the time of the release of that distribution and therefore it would be inappropriate for any earlier (or later) implementation to be advertising itself as providing java
to the system. They would like Temurin to only provide java-21
and similar and not a default java
implementation from the package manager's perspective.
To see the concern in action, running rpm -q provides java
on a recent Fedora system (e.g. 40 - if you're using something pre-dnf then yum whatprovides java
will give the same information) will show this - note that the java-17-openjdk
and other earlier versions are not shown in this list:
java-21-openjdk-1:21.0.2.0.13-3.fc40.aarch64 : OpenJDK 21 Runtime Environment
Repo : fedora
Matched from:
Provide : java = 1:21.0.2.0.13-3.fc40
If you add in the Temurin repositories there will be entries for all of the different versions available from Temurin, which is in conflict with the Fedora requirements.
Some notes:
java
in /etc/alternatives/java etc. but will be overridden if you subsequently install the "system" JDK of 21. By contrast if you have, for example, Fedora's JDK17 installed and then install temurin-8-jdk the the default java
will become Temurin 8 and attempts to run dnf install java
will not result in JDK21 being installed (because temurin-8-jdk Provides: java
), which is likely to break dependencies for products expecting java
to be 21 on that distribution. Fundamentally it's a problem with consistency.Provides: java
may be different so just saying "The latest LTS can have Provides: java
" is not appropriate as a solution here. For example on UBI9 it is java 11, and while they have later versions in the repository they do not have Provides: java
and java 11 overrides 17 as the default on that distribution. Currently Temurin will always override the default regardless of which version is installed because it Provides: java
is_system_jdk
variable in their spec files e.g. java-17 and java-21java
based on this value:
Provides: java-%{origin}%{?1} = %{epoch}:%{version}-%{release}
Provides: jre-%{origin}%{?1} = %{epoch}:%{version}-%{release}
Provides: java%{?1} = %{epoch}:%{version}-%{release}
Provides: jre%{?1} = %{epoch}:%{version}-%{release}
%endif
The concern from a Temurin perspective is that this cannot be replicated with a single RPM (Since the distribution determines which one is the "system" JDK and Temurin currently does not provide multiple RPMs for each distribution and I think we we would prefer not to start doing so, especially since we'd rather reduce the complexity of our uploads as per https://github.com/adoptium/installer/issues/501
Temurin does not currently have a Provides
for java-11-openjdk to work as a drop-in replacement (but we do provide java -11
so if there was a "shell" (metapackage?) java-11-openjdk package (and maybe even java
in a distribution-specific way) going forward that could be fulfilled by java-11
from Temurin that would allow it to be a drop-in replacement, other than the fact we still have Provides: java
(Not, that changing it to Provides: java = 21...
would not resolve the concern based on this comment)
While it's not directly related to the above, we should consider listing our Provides
lines with a specific version number in the Temurin spec files e.g. Provides: java-11 = 1:21.0.2.0.0.13-2
It has been noted that other vendors like Amazon, Azul, and Red Hat, have explicit versions for their RPM's Provides fields.
Noting also that Temurin currently provides separate rpm files in the repository for RHEL and SUSE. One option here would be to have a third option for Fedora with an alternative setup (which would remove any backwards compatibility concerns for Temurin users, although as noted above UBI (And therefore presumably RHEL) would have the same concern so adjusting for the existing rhel
version may be appropriate. The difference being that Temurin is not being considered as the default implementation for anything other than Fedora.
true
* The "system" JDK version in Fedora is done with the `is_system_jdk` variable in their spec files e.g. [java-17](https://src.fedoraproject.org/rpms/java-17-openjdk/blob/rawhide/f/java-17-openjdk.spec#_94) and [java-21](https://src.fedoraproject.org/rpms/java-21-openjdk/blob/rawhide/f/java-21-openjdk.spec#_94) * There is a section in the spec files that choose whether to provide `java` based on this value:
Provides: java-%{origin}%{?1} = %{epoch}:%{version}-%{release} Provides: jre-%{origin}%{?1} = %{epoch}:%{version}-%{release} Provides: java%{?1} = %{epoch}:%{version}-%{release} Provides: jre%{?1} = %{epoch}:%{version}-%{release} %endif
The concern from a Temurin perspective is that this cannot be replicated with a single RPM (Since the distribution determines which one is the "system" JDK and Temurin currently does not provide multiple RPMs for each distribution and I think we we would prefer not to start doing so, especially since we'd rather reduce the complexity of our uploads as per #501
I would be actually quite happy to include this variable and its usage into Temrun's specfiles. Only it will remain for ever false.
I would be actually quite happy to include this variable and its usage into Temrun's specfiles. Only it will remain for ever false.
🤔 I quite like that idea ... It would also give anyone else who wanted to the ability to create an rpm using our scripts with the option turned on if they wanted to. (I guess it could also be a list that has the version(s) that can act as system ones but that's maybe more complex than would be required). I think from the Adoptium perspective at this point it's a question of "Are we happy with this being off by default?" and go from there.
(And with that, I'm shutting the laptop down until June! See you all next month)
think from the Adoptium perspective at this point it's a question of "Are we happy with this being off by default?" and go from there.
Exactly. Happy vacation!
I had created all pinpointed subissues:
@jiekang @tellison fyi!
Temurin SDKs as non-system JDKs in Fedora
The goal is to get rid of redundant maintenance of legacy JDKs in Fedora. And to keep legacy jdk support via temurin repositories https://adoptium.net/installation/linux/
This may be prequel to doing similar in RHELs
Prequel
JDK8 is slowly but surely disappearing. On enforced bytecode compatibility level, jdk21 - current system jdk in fedora - is the last one which supports jdk8 bytecode level. For jdk22 minimal bytecode level corresponds to jdk9. Next system jdk - 25 will be bytecode level compatible with jdk12. In addition, jdk11 and 17, do not carry the burden of backward compatibility as jdk8 does, and we can happily claim jdk21 would do.
All distribution people around OpenJDK have slowly moved towards the current reference implementation - Eclipse Adoptium Temurin JDK. Eclipse foundation is providing ecosystem for all possible distributions of JDK, from rpms and deb packages, over portable static tarballs to containers. As it is, it is win win for both users and developers. That includes also support tools like javaws or mission control (openjfx?)
Since jdk21, there is no room for any package to depend on jdk8, 11 and in year or so also on 17.
Thus we would like to continue to maintain future LTS - all STS java packages in rolling java-latest-openjdk, system JDK, and for short transition time previous STS (now 17), but we would lijke to drop all older jdks. Where there should be no lost for users, for developers, a comfortable
dnf install java*
is quite cool feature, and thus we would like to support this via eclipse temurin rpm repositories as an counter weight to SDKMAN and similar tools.Known showstoppers
Known regressions
Proposed offering to fedora 41 system wide change
We will get rid of all non-system fedora/rpms/java-xy-openjdk once they stop being system JDK. As a start, we will discontinue JDKs 8 and 11 as follows:
This decision will flow up from the fedora-devel list and from fesco decision. If both will be strongly for the opinion (1), and indeed a strong maintainer would step in, and will prove themselves, then then we can add Temurins as secondary JDKs. This will be part of the proposal.
(3) is the preferred way to go, but we are not yet there.
To install 3rd party repository have exact rules: