[releng] Several examples plugins don't set a version range on their dependent plugins

| --- | --- | | Bugzilla Link | 335459 | | Status | CLOSED FIXED | | Importance | P3 normal | | Reported | Jan 26, 2011 10:50 EDT | | Modified | May 29, 2012 13:23 EDT | | Reporter | Adolfo Sanchez-Barbudo Herrera |

Description

In principle I'll do a superficial review of the dependencies of every examples plugin so that:

Every plugin sets a minimum version range on its dependent bundle. If there is no anyone I'll set the correspondent version of the bundle in the Helios release.
Every plugin sets a maximum version range on its dependent bundle. Specifically, the next major version (excluding said version).

I'll create a patch for a quick commit for a releng purpose, a deep analysis should be done to tight the dependencies.

By Ed Willink on Jan 26, 2011 12:07

Change whatever you need to get the build to work, but

I try to keep the limits a bit loose particularly on 'stable' / 'compatible' dependencies such as platform and UML2. I try to let EMF impose its limits, since EMF is a singleton, if you've got a compatible EMF+platform combination let's try to work with it.

Xtext clearly needs a 2.0 lower bound, but Xtext 2.0 is intended to work on Helios so it would be nice to minimise what doesn't. ocl.ecore has an extension point only dependency on EMF M4, so should work backwards. Only the OCL console has a code dependency on EMF M5.

By Adolfo Sanchez-Barbudo Herrera on Jan 26, 2011 13:14

Ed,

I know that setting the dependencies is usually complicated. Neither I'm expert enough in the examples the code nor I'm a releng guru to have magic recipe to efficiently/precisely do that.

The only rule I had in mind is that is good setting a version range. However, if they are not set from the beginning It's probably quite hard to know which version should be the minimum EMF 2.6/2.5/2.4/ for instance.... I don't really have time to analyse the code and the functionality of every examples plugin to find a proper EMF/UML/Xtext + platform version range combination for every plugin/bundle on which we depend...

I could say that some editors (for instance OclInEcore one) should probably set EMF 2.6 as minimum version due to the delegates. Now, the Console should probably need a minimum EMF 2.7 due to the new EMF Query Delegates feature...

So, If you have been sensibly using a policy to set the version ranges of every dependency of our plugins is Ok to me. Nevertheless, it could be good to contrast it with other experienced eclipse plugin developers to know if we are doing right...

Concerning the Indigo's Build... I could disable the examples feature until the proper Platform/EMF/UML/Xtext repositories are setup...

For anyone interested, our Indigo examples' contribution is failing because:

Eclipse platform releases a new org.eclipse.core.runtime 3.7 bundle in next milestone (M5), but the current indigoRun is configured to use the platform milestones repository which contains the previous 3.6 one.
We are providing examples plugins which depends on the new org.eclipse.core.runtime 3.7 bundle. Our integration build didn't fail because we use the Eclipse Platforom I-builds Repository which indeed contains the new org.eclipse.core.runtime 3.7 bundle.
Our building process is probably setting the 3.7 minimum version range for some plugin's dependencies because:
- There is no any range version in their dependencies.
- It obtains the range version from the bundle in the target platform. Since we are using the Eclipse platform I-builds repository as our target platform, a [3.7.0, 4.0.0) range version is set for every plugin's dependency which doesn't have a range version.

I'll temporarily disable the Indigo MDT/OCL Examples' contribution,

Best Regards,\ Adolfo.

By Adolfo Sanchez-Barbudo Herrera on Mar 23, 2011 12:58

Sorry for the long post but I have some interesting conclusions which I may gather from my experience dealing with plugin versions:

When setting a version match of a plugin on which we depend, we have three possibilities:\ a) No setting any version.\ b) Setting an specific version match (setting minimum or marximum version in the plugin-manifest editor).\ c) Setting a version range match (setting minimum and maximum version in the plugin-manifest editor).

We could discard now option b). Otherwise, with every release, we would have to manually change the depencency's version of every modified plugin on which we depend.

In principle, option a) didn't look a good idea because we can't say that our plugin work with every version of a plugin on which we depend. However, in our case ,Buckminster/PDE automatically generates a version for this dependency in our plugin manifests... Which version is used? The answer is the specific version of the bundle which is present in the target platform used by our build.

The option c) has been always a good approach but, as I will explain later, has some inconveniences. Traditionally, when I add a new dependency on a plugin, I set the minimum version range using the available version in the current target platform (usually the running Eclipse instance) and I set the maximum version range as the next major version of said plugin. I do this because our plugin should work with new versions of our depending plugin provided no major version increment occurs (due to Eclipse API policies). The Eclipse API policies provides us of some kind of forward compatibility.

For instance, if I have a plugin X whose version in my Eclipse is 2.3.1, and I'm developing a new plugin which depends on it, I'll probably set the following version range on the dependency [2.3.1,3.0.0).

Being pedanticly rigorous we are making the first mistake... Why a 2.3.1 minimum version range ?. Why not 2.3.0 ? why not 2.2.0 or 2.0.0 ?... Well, at this point, we are lazily practical and we don't want to spend time in trying to find out the real minimum version of the plugin on which we depend which could make our own plugin work, just because at least our plugin will properly work with X 2.3.1 .... Well, a minor issue, who cares, who will try to make our current plugin work with previous releases ?... let's say no problem so far.

However, after a new release comes up, we start a new development stream, that is, new plugins to develop, new plugin on which we will depend on, we will probably do modifications on our plugins, and of course plugins on which we depend will do their modifications as well.... Now, what happen with the dependency of the plugin X which will probably now be the plugin 2.4.0 ?. Well, we could probably ensure that our plugin properly works with the plugin X 2.4.0 since we are continuously developing and testing against it, but are we sure that our current plugin still works with the plugin X 2.3.1? The answer is definitely NO... Well perhaps if our plugin doesn't change will do really work for sure... but, for instance, as soon as we use in our plugin some new API incorporated in the plugin X 2.4.0 our dependency version range on that plugin is WRONG because our plugin won't simply work with it.... we must definitely increase our minimum version range to 2.4.0.

The point is that we don't are not doing that (at least I don't usually it) so we are incrrectly managing our dependencies. In other words we are offering/defining some kind of backward compatibility which is not really true.

As far as I understand the problem we have two options when dealing with version ranges:

c.1. Either we increase the minimum version range of every plugin on which depend on when they change its version (a similar inconvenience to option a) )\ c.2. or we provide some kind of testing process in which we ensure that our plugin work with a previous release of said plugin (which may allow us to know the exact version of our dependency, so that we may establish our backward compability boundaries). Besides, this would also require the revision of our dependencies.

As you might imagine, when dealing with something more complex than a simple plugin dependency of a simple plugin, trying to achieve c.2 doesn't seem an easy task....

So we have the following solutions:

b) No setting any specific version match on our dependencies\

Simple to achieve. Just remove the version range of our dependencies on every plugin.\
No problems of maintainability of our dependencies.\
The version of our plugin dependencies will be always correct.
We don't have any backwards compatibility.
We don't have any forwards compatibility (Our releases will only work with the official Platform release).

c.1) \

Simple to achieve. Just adjust minimum version ranges of our plugins.
Harder maintainability and error prone (we may forget to change this minimum version range or depending plugins may silently change its version without noticing it)\
The version of our plugin dependencies will be always correct (provided we don't miss any minimum range version update of our plugin dependencies :) )
We don't have any backwards compatibility.\
We will have forwards compatibility

c.2)

Hard, if not impossible to achieve. \
Good maintainability, providing that our testing process were good enough to detect problems when testing againts previous versions.\
The version of our plugin dependencies will be always correct.\
We would have backward compatibility, and we will know the minimum requirements to make our componentes work.\
We would have forward compatibility.

d) A new option: Do nothing right now :)\

Easy to achieve. Do nothing :) (well, maybe adjusting some ranges in the some deficient plugins).\
Good maintainability. Nothing to do, excepting adjusting minimum version range when depending plugin make a major version change.
The version of our plugin dependencies may be incorrect.\ ? We would have some kind of backward compatibility, but we wouldn't know which versions of our depending plugins our components could work with\
We would have forward compatibility.

My preferences:

On the one hand, in spite of being the best/ideal solution, I discard c.2 because just imagining how to find out the specific dependency of plugins on which we depend on, it makes me fear. However if somebody want to efficienctly find and implement a solution for c.2 it will be very welcome.

On the other hand, I'm inclined to do the solution A) giving the following conclusions:

Is forward compatibility is really crucial ? (why may I want to use Helios MDT/OCL in a Indigo release ?).
Is having some kind of unknown backward compatibility really better than saying that we don't offer backward compatibility ?.
I really consider that specifiying incorrect dependencies is no acceptable.

Please, let me know if this makes sense and/or your opinion.

Best Regards,\ Adolfo.

By Adolfo Sanchez-Barbudo Herrera on Mar 23, 2011 13:04

(In reply to comment #3)\

So we have the following solutions:

b) No setting any specific version match on our dependencies

Simple to achieve. Just remove the version range of our dependencies on every plugin.

No problems of maintainability of our dependencies.

The version of our plugin dependencies will be always correct.

We don't have any backwards compatibility.

We don't have any forwards compatibility (Our releases will only work with the official Platform release).

Apologies. This one, is the a) option (my choice) ...

the b) option was discarded in the beginning of the comment.

Cheers,\ Adolfo.

By Adolfo Sanchez-Barbudo Herrera on Mar 23, 2011 13:07

(In reply to comment #2)

c.1. Either we increase the minimum version range of every plugin on which depend on when they change its version (a similar inconvenience to option a) )

Oh dear !!!

I obviously meant "(a similar inconvenience to option b) )"

So, I'd also discard this one....

Sorry very much for the confusion >.<

Best Regards,\ Adolfo.

By Ed Willink on Mar 23, 2011 16:00

I agree it's difficult. Traditionally the policy is only to support new code on current and future platforms. In practice this is imposed by EMF which tends to do something to prohibit the previous release at about M2. So I have been happy to just have an exact EMF lower bound somewhere and leave everything else generous.

However Acceleo's usage goes beyond this, so I would like to understand how Acceleo support Helios OCL on Ganymede, so that any decision we make does not make Acceleo's compatibility harder, and possibly makes it easier.

Checking out HEAD on Ganymede shouldn't be too hard and if the JUnit tests are green, I think that counts as tested on Ganymede. We could have Hudson jobs that did this automatically, since all the project dependencies are frozen, this job should not need maintenance.

By Laurent Goubet on Mar 24, 2011 04:59

(In reply to comment #6)

However Acceleo's usage goes beyond this, so I would like to understand how Acceleo support Helios OCL on Ganymede, so that any decision we make does not make Acceleo's compatibility harder, and possibly makes it easier.

Checking out HEAD on Ganymede shouldn't be too hard and if the JUnit tests are green, I think that counts as tested on Ganymede. We could have Hudson jobs that did this automatically, since all the project dependencies are frozen, this job should not need maintenance.

I'll try not to get this to be a too bothersome comment... but I don't really know how to explain Acceleo compatibility choices shortly, so please bear with me :p.

Acceleo's choice is ... to leave the choice of their platform to the users. "Big" industrial users tend to deploy one version of Eclipse on many computers, and to stick with this version unless absolutely forced to move on by one project or another for which they cannot afford to pass a new version by (fixes a critical bug for them, and either 1) the project does not accept to backport the fix or 2) the price to backport the fix is higher than the price to upgrade all machines to the new Eclipse version).

This is somewhat of a simplification of the real use case, but I believe it to be as close as possible to reality.

For projects that aim at such users, there can be a number of choices :\ a) offer long-term support, maintenance and evolution contracts\ b) refuse to maintain old versions\ c) maintain compatibility of your newest versions with "old" releases of your dependencies\ d) a mix of the three first

Acceleo chose d), mixing a) (for clients who don't want evolutions brought by the newest Acceleo) and c) (for clients who wish to be able to use the new and improved Acceleo in older Eclipses).

That said, we do not "support Helios OCL on Ganymede" : Acceleo supports "all versions of OCL since Ganymede in their respective platform". When Acceleo 3.0.2 (released for Helios SR2, against EMF 2.6.2 and OCL 3.0.2) is installed in Ganymede, it will leverage EMF 2.4.x and OCL 1.2.x. When it is installed in Galileo, it will make use of the enhancements brought by EMF 2.5 and OCL 1.3. When used in Helios, it is in its dedicated environment, and will thus leverage the features of EMF 2.6 and OCL 3.0.

That is what I meant when I told about "compatibility layers" : we have code in Acceleo that targets specifically Helios, code that target specifically Galileo ... and code that makes sure that this "version specific code" is never called when using older Eclipse versions. That is : Acceleo does not compile entirely in Ganymede or Galileo! However, the code is compiled against the latest Eclipse/EMF/OCL, and since the parts that would not compile in such or such Eclipse are never called at run time in these Eclipses... we are compatible. This is also the reason why we have limitations such as "Acceleo code compiled in Helios might not be runnable in Ganymede" : as we leverage the newest OCL version when compiling the code, we cannot be sure that all compiled artefacts will be evaluate-able by an older OCL. This might sound obscure, but I don't really know how to explain this otherwise :).

Thus, Acceleo "lower version bounds" are the oldest we maintain compatibility towards, and they do not evolve as long as we do not decide to break anything. We've specifically disabled the buckminster things that mingle with our MANIFEST.MF and put "hard" dependencies towards the specific versions that were available at build time.

As for how we test this : we always code with a number of target platforms : 3.4, 3.5, 3.6 and 3.7 (i.e "the current target"). We usually code against our latest target, making sure not to use APIs that are defined "@since " unless in "guarded" code (that will only be invoked on the specific condition that this dependency version is available). We switch to our other target platforms from time to time -usually before all milestones- in order to launch all of our test suites.

As you can see, this is not an easy endeavor. We made this choice in Acceleo as we believe backward compatibility to be of utmost importance, we will encourage other to try and do the same ... but we won't try and force them. Furthermore, we have come to expect them not to try and be compatible, and thus code protectively ourselves. OCL 3.1 not being compatible with Helios is no problem for Acceleo, OCL 3.1 breaking some API isn't either (though the more API breakage, the more difficult for us).

My recent concerns were only for compatibility with the 1.5 VM, as we have no way to "code protectively" if OCL decides to use 1.6-specific code other than fork that part and turn it into java 5 compatible code (if at all possible).

PS: As I thought, didn't manage to make a short, readable comment... sorry about that if you read this far :).

By Adolfo Sanchez-Barbudo Herrera on Mar 24, 2011 06:41

Laurent,

You post is very helpful, so thanks a lot for it. The most important thing is that considering any MDT/OCL backward compatibility support should not impact Acceleo one... Good ;)

The point is that we don't know what to do with backward compatibility from MDT/OCL point of view, yet, and how are we going to prove and manage said compatibility support.

It looks like you have been doing a great job concerning backward compatibility, and it doesn't seem to be a "trivial" thing. Discarding code execution depending on the platform the code is being executed, needs at least some extra thoughts and consideration during development (Fortunately, you decided to use OCL Facades, rather than extending OCL grammars ;P )...

Anyway, I think we should try to fix (if case they are not fixed) our dependencies... and to do that we must need a consensous about "Do we want to offer backward compatibility?"...

Again, the same 4 solutions are on the table to fix this and Bug 340748, if they are not clear enough or we require another solution, just let me know...

Laurent, I have some in-lined comments below

(In reply to comment #7)

This is also the reason why we have limitations such as "Acceleo code compiled in Helios might not be runnable in Ganymede" : as we leverage the newest OCL version when compiling the code, we cannot be sure that all compiled artefacts will be evaluate-able by an older OCL. This might sound obscure, but I don't really know how to explain this otherwise :).

Could you specify what you mean with "Acceleo code"?. I guess you don't mean the Java classes from Acceleo project, but Acceleo templates or something like that ?

NB: I confess I've not played with Acceleo too much\

Thus, Acceleo "lower version bounds" are the oldest we maintain compatibility towards, and they do not evolve as long as we do not decide to break anything. We've specifically disabled the buckminster things that mingle with our MANIFEST.MF and put "hard" dependencies towards the specific versions that were available at build time.

I think that buckminster does that only with dependencies which doesn't specify a version match, am I wrong ?

As for how we test this : we always code with a number of target platforms : 3.4, 3.5, 3.6 and 3.7 (i.e "the current target"). We usually code against our latest target, making sure not to use APIs that are defined "@since <something too recent>" unless in "guarded" code (that will only be invoked on the specific condition that this dependency version is available). We switch to our other target platforms from time to time -usually before all milestones- in order to launch all of our test suites.

Yes this is more or less, what I had in mind with c.2 option. Focusing on target platforms (Ganymede, Galileo, Helios, ...) rather than trying to figure out the minimum version range of dependencies looks more plausible. If tests cases doesn't pass using Ganymede installation, we are not Ganymede compatible, then try Galileo, etc...This sounds better than nothing... trying to automate this looks like needing extra efforts... Would we need to also establish backward compatibility distinctions between MCT/OCL Core and Tools (editors, console, etc) ? Probably...

My recent concerns were only for compatibility with the 1.5 VM, as we have no way to "code protectively" if OCL decides to use 1.6-specific code other than fork that part and turn it into java 5 compatible code (if at all possible).

As Ed finally proposed in mdt-ocl dev list, I guess we won't change the Java Execution Environment requirement.

Best Regards,\ Adolfo.

By Laurent Goubet on Mar 24, 2011 07:55

(In reply to comment #8)

It looks like you have been doing a great job concerning backward compatibility, and it doesn't seem to be a "trivial" thing. Discarding code execution depending on the platform the code is being executed, needs at least some extra thoughts and consideration during development (Fortunately, you decided to use OCL Facades, rather than extending OCL grammars ;P )...

Extending OCL grammars as been discarded very early in the development. As I mentionned above, "we believe backward compatibility to be of utmost importance". This was already true when we started developping Acceleo 3.

Anyway, I think we should try to fix (if case they are not fixed) our dependencies... and to do that we must need a consensous about "Do we want to offer backward compatibility?"...

I am not active enough (okay ... not at all) in OCL to really tell you to go one way or the other here. Doing what we do in Acceleo is what I believe to be the best option, yet it does impact development : we always have to be extra careful not to use the latest functionality if we cannot provide compatibility workarounds. For example, we haven't been able to leverage the "MinimalEObjectImpl" EMF provides since 2.5 because of these compatibility issues; and we won't until we decide to drop Ganymede support.

If you do not wish to have extra worries when developping, then you shouldn't aim to provide backward compatibility (after all, EMF itself does not ...). Of course, you taking those steps would reduce the strain on our (Acceleo) development as it would make one less dependency to react to breakages in (humm ... not sure this was proper english :D).

Could you specify what you mean with "Acceleo code"?. I guess you don't mean the Java classes from Acceleo project, but Acceleo templates or something like that ?

What I meant by "Acceleo code" was indeed the Acceleo templates. Acceleo implements the MOF Model-to-text Transformation Language OMG specification (whew, talk about buzzwords). This specification is a specification for a template language to write code generators. "Acceleo code" referred to the language and how we "compile" it to EMF models.

I think that buckminster does that only with dependencies which doesn't specify a version match, am I wrong ?

Nope, you're right. But we don't specify version ranges for some of our dependencies, and were thus hindered by this.

then try Galileo, etc...This sounds better than nothing... trying to automate this looks like needing extra efforts... Would we need to also establish backward compatibility distinctions between MCT/OCL Core and Tools (editors, console, etc) ? Probably...

We haven't automated this for Acceleo. As mentionned above, the code does not compile in Ganymede or Galileo : building with Helios (or greater) is mandatory. We thus manually switch the target platform, make sure that there are no more parts that do not compile than we expect, then launch the unit tests to make sure these portions of code are indeed never called, and the unit tests are all green.

As Ed finally proposed in mdt-ocl dev list, I guess we won't change the Java Execution Environment requirement.

Changing the execution environment isn't a problem, so long that the compiler is set to java 5 compatibility. It is the build environment that matters ;).

By Ed Willink on Mar 24, 2011 08:17

Thank you Laurent; that was very enlightening.

I think that for tooling we are agreed.

Indigo develop with 6, binary compatibility 5, minimum JREE 5.\ Juno develop with 7, binary compatibility 5, minimum JREE 5.

For our code there are three kinds of API compatible enhancement:

Enhance to provide something new\

Not a problem. @since/our new version enables consumers not to consume it.

Enhance to consume something new\

This is what really worries me. The very small change to support query delegates makes almost the whole project uninstallable on Helios.

I am strongly inclined to migrate the query delegates to a ....indigo plugin. Please comment on this idea in Bug 340844.

Enhance to do something differently\

This may always be difficult, because any bug fix causes something to be different. For instance OclInvalid/invalid was clearly progress but also different. If we can make the code backwards and forwards runnable, then the exact functionality is the reason for choosing a particular version, so if a consumer wants version X to behave as version Y then that is a consumer problem. If you want Y behaviour, use release Y, which we try to make runnable on a wide range of platforms.

By Adolfo Sanchez-Barbudo Herrera on Mar 24, 2011 09:14

(In reply to comment #10)

This is what really worries me. The very small change to support query delegates makes almost the whole project uninstallable on Helios.

Ed,

As far as I've learnt from the this thread, I'm not sure "uninstallable" is the best word... As long as we have EMF 2.6 as the minimun version in our dependencies, our plugins would be actually installable... The problem is that as soon as the new EMF 2.7 API is called, the user will probably get some kind of RuntimeException...

Acceleo avoids this kind of malfunctioning simply checking the current running platform before calling the new API. I guess it's something like this:

If (is_running_emf_2_7()) {\ // Use the new EMF 2.7 API ...\ }

This is the extra effort to consider while depeveloping if we want to keep backwards compatibility.

I'll have to think a little bit more on your alternative and its implications, which I'll post on the new raised Bug 340844.

Again we need to balance the benefits and inconveniences for clients and developers to obtain a good trade off to decide if we want backwards compatibility and to know against which platform we support said backwards compatibility ...

Cheers,\ Adolfo.

By Adolfo Sanchez-Barbudo Herrera on May 06, 2011 09:00

Ed,

I've reviewed all the examples plugins and:

either they have a version range (minimum and maximum) on its dependencies.
or they don't have any version to match.

So, in principle, we don't need to do any change to them right now... Anyway, it doesn't seem that we have agreed about what to do with our dependencies and how to test that our dependencies ranges are right, how we are providing compatibility with previous releases, or even if we are want to provide said compatibility, etc

Let me know if you want to close this bugzilla or keep it open for the future...

Cheers,\ Adolfo.

By Adolfo Sanchez-Barbudo Herrera on May 06, 2011 10:15

Sorry,

I didn't have o.e.o.e.impactanalyzer plugins in the current workspace.

o.e.o.e.impactanalyzer.ui sets a dependency specific version match on o.e.ocl.ecore plugin.

Since every plugin's dependency don't have any version match, I'm removing it...

Changes committed to HEAD.

By Adolfo Sanchez-Barbudo Herrera on May 10, 2011 07:22

I think it's better to close this bugzilla since we have a similar Bug 340748 in which we are keeping the same discussion (which includes all the plugins rather than the examples one).

Resolving as fixed (there is no any examples plugin dependency with specific version match).

By Ed Willink on May 29, 2012 13:23

Closing all bugs resolved in Indigo.

eclipse-ocl / org.eclipse.ocl