org.eclipse.equinox.http.service.api and org.osgi.service.http.whiteboard not available in Equinox SDK v4.31 zip archive

[mbaudier]

Our runtime is based on Equinox and we use the Equinox SDK zip archive for provisioning.

After the deprecation of org.eclipse.osgi.services (cf. https://github.com/eclipse-equinox/equinox/pull/403), the replacement bundles org.eclipse.equinox.http.service.api and org.osgi.service.http.whiteboard are not available in the released archive: https://download.eclipse.org/equinox/drops/R-4.31-202402290520/download.php?dropFile=equinox-SDK-4.31.zip

It is not clear to me whether this is on purpose (cf. https://github.com/eclipse-equinox/equinox/issues/487), but since some of the provided bundles depends on them (for example org.eclipse.equinox.http.servlet) and that they are rather small, it would nice to have them in the archive, along with their sources.

[merks]

If I understand correctly, this product's contents determine the contents of the zip:

https://github.com/eclipse-platform/eclipse.platform.releng.aggregator/blob/master/eclipse.platform.releng.tychoeclipsebuilder/equinox-sdk/equinox-sdk.product

I assume that based on this comment:

https://github.com/eclipse-platform/eclipse.platform.releng.aggregator/blob/9bb9ff8da096e2d2f3bd2875fb90c7347a4234a4/eclipse.platform.releng.tychoeclipsebuilder/equinox-sdk/pom.xml#L12-L13

I expect a product to include the closure of all requirements but your comment suggests that's maybe not the case, which is confusing to me. Or maybe you are suggesting the deprecated thing is included rather than then non-deprecated things.

If things need to be added, they need to be added to the product or required by some feature or plugin used by the product...

[HannesWell]

If things need to be added, they need to be added to the product or required by some feature or plugin used by the product...

I think that's what he is suggesting, but a confirmation would be good.

[mbaudier]

If, as is suggested, the zip archive creation is driven by products/features, and since org.eclipse.osgi.services is listed in this feature: https://github.com/eclipse-equinox/equinox/blob/master/features/org.eclipse.equinox.core.feature/feature.xml I guess its "replacements" org.eclipse.equinox.http.service.api and org.osgi.service.http.whiteboard should be listed there as well, along with the other OSGi APIs. Right?

[laeubi]

All dependencies should be included automatically. So if this is not happen here it should better be investigated why it is the case. Also it seems not the best to provision a product based on this zip file, why are you not using Tycho / P2 to provision your runtime from the update sites directly?

[merks]

I suppose one gotcha when a bundle has a package requirement satisfied by more than one other bundle, it’s not obvious which one will bundle will be chosen as part of the set of all dependencies.

[laeubi]

I suppose one gotcha when a bundle has a package requirement satisfied by more than one other bundle, it’s not obvious which one will bundle will be chosen as part of the set of all dependencies.

But then how are things can be missing?

[merks]

Missing seems subjective. If a deprecated bundle is present but the preferred alternative is absent what can one say about that? Is something missing?

In any case if there are bundles that we want people to use as an alternative to bundles we’d rather people not use, we probably want to encourage that by ensuring they are present rather than absent.

[mbaudier]

Missing seems subjective. If a deprecated bundle is present but the preferred alternative is absent what can one say about that? Is something missing?

The org.eclipse.osgi.services bundle does not contain any more the Java packages for OSGi HTTP service (legacy) and OSGi HTTP whiteboard. For example:

org.osgi.service.http
org.osgi.service.http.context
org.osgi.service.http.whiteboard
...

This is what is missing, and would now be provided by the org.eclipse.equinox.http.service.api and org.osgi.service.http.whiteboard bundles.

[laeubi]

@merks at best we would just remove the services bundle but the SDK zip is considered legacy/api we can't break. Thats why I asked how one "provisions" from the Zip as it not necessarily contains 'everything' that's useful/recommended.

@mbaudier can you tell what bundle in the zip exactly depends on the mentioned packages?

[mbaudier]

@merks at best we would just remove the services bundle but the SDK zip is considered legacy/api we can't break. Thats why I asked how one "provisions" from the Zip as it not necessarily contains 'everything' that's useful/recommended.

While we use the P2 Director when building our Eclipse plugins or a complete IDE, Equinox as such is used for a lightweight low-level runtime which is bootstrapped with minimal dependencies (not even Maven). We have other mechanisms to retrieve artifacts (which we are using to workaround this issue), but this is useful to start with a clear reference archive where we pick and choose what we need with simple glob patterns. It has been many years that it just works, and that we don't have to think about it.

But the question is rather whether the SDK zip file is still supported? If not, this is not a big deal, and we will proceed differently at some point.

@mbaudier can you tell what bundle in the zip exactly depends on the mentioned packages?

As I wrote above, we actually only use a subset of the zip, so I just did a quick test and at least the following bundles from the Equinox SDK do not resolve without org.eclipse.equinox.http.service.api and org.osgi.service.http.whiteboard :

28  INSTALLED   org.eclipse.equinox.http.jetty_3.9.100.v20240213-1244
38  INSTALLED   org.eclipse.equinox.http.servletbridge_1.3.0.v20240213-1244
60  INSTALLED   org.eclipse.equinox.http.registry_1.4.0.v20240213-1244
75  INSTALLED   org.eclipse.osgi.services_3.12.0.v20231218-2126
153 INSTALLED   org.eclipse.equinox.http.servlet_1.8.100.v20240213-1057

(It also breaks Eclipse RAP's OSGi integration

52  INSTALLED   org.eclipse.rap.rwt.osgi_3.28.0.20240110-1440

which we pick up from their archive, where these bundles are also missing, but I thought that I would check here first before raising the issue to them, as this is probably related)

[laeubi]

I now looked into this and at least here it says to not include all dependencies:

https://github.com/eclipse-platform/eclipse.platform.releng.aggregator/blob/9bb9ff8da096e2d2f3bd2875fb90c7347a4234a4/eclipse.platform.releng.tychoeclipsebuilder/equinox-sdk/pom.xml#L44

that means that the product might contain whats required, but to me it looks like the repository is archived.

But the question is rather whether the SDK zip file is still supported?

Good question, at least it seems there is no automatic validation then, given todays complexity in dependency chains it at least looks hard to manage manually. Given that we don't want to maintain third party dependencies in features anymore such problems can arise more often.

To "fix" the problem one might want to switch to a "mixed" product and add the missing bundles there to have a hard requirement on them. Other options include using the director mojo together with individual target platform extra dependencies.

[merks]

I'm not sure the reason for why not to include all dependencies, other than that there are a lot of them!

Given the extent to which includes of 3rd party dependencies are being removed from features, I think not including all dependencies leaves the repository such that it is very much not self contained.

I did the following experiment using the CBI aggregator. I defined an aggregation that includes everything from the Equinox SDK, and only one specific bundle from the 4.31 Platform SDK update site:

The aggregation then includes the full set of requirements of the Equinox SDK, including also ones in the Platform SDK that are required by the Equinox SDK. That's a really huge list:

So one could argue that all of them are missing, not just the two mentioned in this issue.

@tjwatson

The Equinox SDK is quite old and I really don't understand its design and purpose. Should we include all requirements in it? Should we strop producing it?

[laeubi]

@merks yes thats what I mean, the zip is more about downloading "everything from equinox" but not necessarily contains "everything from equinox and what it depends on".

This might be equal in the past and worked depending on the subset of bundles choose (given that often we have mentioned dependencies in the features in the past) e.g. if you do not select anything ui related, but now is beginning to suffer from bit-rot as we:

1. remove dependencies (or not add them anymore) from feature.xml
2. more and more "externalize" things (e.g. reusing items from the "official" OSGI artifacts)

[tjwatson]

@tjwatson

The Equinox SDK is quite old and I really don't understand its design and purpose. Should we include all requirements in it? Should we strop producing it?

I've probably either forgot the original design and purpose or, maybe more likely, never really "got it" for how the Equinox SDK was "designed". Originally I wanted Equinox to be the "core" which had no dependencies on anything from the rest of the Eclipse project (UI, runtime etc.). But that vision never came to be. Too much got added to Equinox that I think never belonged. For example UI components of Security. This should never have been added to Equinox, it should have been part of platform to avoid circularity issues. I have similar thoughts on p2, which I think should have been its own sub-project of Eclipse or just part of the Platform, like the old Eclipse Update. There are many other things that I would have preferred to never have been added to Equinox (looking at you bidi bundle and you transforms).

But since Equinox evolved into more it became harder to keep the SDK zip to be fully resolvable in isolation. Instead it became a convenience artifact for downloading all the equinox bundles. Today I only ever use the Equinox SDK so I can set my API tools target easily to include all the Equinox bundles locally since they are not all included in the Eclipse SDK.

Besides that, I've no need for the Equinox SDK zip personally. But like many other old things we have, I am nearly certain someone has a dependency on this thing existing.

[merks]

So if I change it to include all requirements probably no one will complain and we can avoid too much deep thought about it…

[tjwatson]

So if I change it to include all requirements probably no one will complain and we can avoid too much deep thought about it…

If that makes life easier then go for it.

[laeubi]

@tjwatson I think things have become more easier today with git(hub) so If you feel something should be moved out of equinox let me know we already have moved things around in the past without much hassle.

[tjwatson]

@tjwatson I think things have become more easier today with git(hub) so If you feel something should be moved out of equinox let me know we already have moved things around in the past without much hassle.

Besides org.eclipse.equinox.bidi and org.eclipse.equinox.security.ui I don't think we have a good existing home to move things to. For these two I would suggest they move to platform. But I'm not sure it is worth your time to make this move. I wont complain if you choose to spend time on it though.

p2 is already in its own github repo, but it contributes to the Equinox SDK and p2 has UI components also so it would pull in a bunch of stuff from platform anyway to make it fully resolvable.