Enable boot delegation for backward compatibility

[kaikreuzer]

Moving from plain Equinox in alpah2 to Karaf in beta1 is causing many problems, specifically with openHAB 1 add-ons, see e.g. https://community.openhab.org/t/error-on-weather-binding-when-starting-openhab2/6081 https://community.openhab.org/t/avm-fritz-box-binding-oh2b1-issue/6321 https://community.openhab.org/t/tellstick-binding-for-openhab2/1582/14 https://community.openhab.org/t/we-need-your-help-on-testing/298/29

and many more. This is very annoying since all of this used to work smoothly before.

Many of the issues seem to be due to the fact that JDK classes such as javax.xml are not on the bootclasspath. I wonder if this can be changed for the sake of backward compatibility (knowing that it is no the "clean osgi way")?

To our Karaf experts @dvanherbergen, @maggu2810, @anierbeck, @cschneider: Do you have a suggestion what we should do?

[ANierbeck]

I'm not really sure I follow the issue here, if it's just about JAXB, how about using the same set of bundles as the cxf jaxb feature.

  mvn:org.apache.servicemix.specs/org.apache.servicemix.specs.stax-api-1.0/2.4.0 start-level=10
  mvn:org.apache.servicemix.specs/org.apache.servicemix.specs.jaxb-api-2.2/2.4.0 start-level=10
  mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.jaxb-impl/2.2.11_1 start-level=20
  mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.jaxb-xjc/2.2.11_1 start-level=20

[maggu2810]

The beta1 release is already out, so all the users will see this failures. I would prefer fixing the bundles to import what they want to use.

I assume there is no problem to add all that stuff to the boot class path, but this will mask corrupt bundles again and perhaps increase the bundle loading time (for every bundle).

For the best user experience you could perhaps add a beta2 that contains that additions and keep the bootclass clean for the next development, but I believe then the bundles get never fixed and such stuff will never be removed from the bootclass path again.

[maggu2810]

Perhaps you could disable that boot delegation for the Eclipse IDE, so the developers will fix their bundles. The manifest first approach will lead to add stuff only if the IDE complains something.

That this approach leads to not removing added and now unused imports is another problem. :wink: And could be addressed perhaps some time later.

[maggu2810]

I tested a custom bundle, adding -Dosgi.compatibility.bootdelegation=false -Dosgi.java.profile.bootdelegation=none to the VM arguments in the Eclipse launch configuration will result in the expected behaviour.

[kaikreuzer]

Ok, this would make the Eclipse runtime to behave more similar to the Karaf runtime. Is there also a way to make it already appear as errors on the project in PDE? This would then be the best way to directly discover those problems (unless the code is in embedded jars).

[kaikreuzer]

@maggu2810 Besides, the imports (which are a rather straight-forward one time effort to fix, I agree), I wonder if issues like https://community.openhab.org/t/error-on-weather-binding-when-starting-openhab2/6081 might also be due to the boot delegation - and we do not have any solution for this at all yet.

[maggu2810]

Yes, it is caused by the boot delegation.

2016-01-17 15:48:58.059 [ERROR] [.w.internal.bus.WeatherBinding:57   ] - Error activating WeatherBinding: unexpected element (uri:"", local:"common-id-mappings"). Expected elements are (none)javax.xml.bind.UnmarshalException: unexpected element (uri:"", local:"common-id-mappings"). Expected elements are (none)
    at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallingContext.handleEvent(UnmarshallingContext.java:726)
    at com.sun.xml.internal.bind.v2.runtime.unmarshaller.Loader.reportError(Loader.java:247)
    at com.sun.xml.internal.bind.v2.runtime.unmarshaller.Loader.reportError(Loader.java:242)
    at com.sun.xml.internal.bind.v2.runtime.unmarshaller.Loader.reportUnexpectedChildElement(Loader.java:109)
    at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallingContext$DefaultRootLoader.childElement(UnmarshallingContext.java:1131)
    at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallingContext._startElement(UnmarshallingContext.java:556)
    at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallingContext.startElement(UnmarshallingContext.java:538)
    at com.sun.xml.internal.bind.v2.runtime.unmarshaller.SAXConnector.startElement(SAXConnector.java:153)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:509)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:380)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl$NSContentDriver.scanRootElementHook(XMLNSDocumentScannerImpl.java:614)

Adding the annotation package (as @cschneider already mentioned) solves this error.

Have seen another error after the first one was solved:

2016-01-17 15:54:27.255 [ERROR] [.w.internal.bus.WeatherBinding:57   ] - Error activating WeatherBinding: CommonId for provider null with id 200, already existsjava.lang.IllegalArgumentException: CommonId for provider null with id 200, already exists
    at org.openhab.binding.weather.internal.parser.CommonIdHandler.addCommonId(CommonIdHandler.java:85)
    at org.openhab.binding.weather.internal.parser.CommonIdHandler.loadMapping(CommonIdHandler.java:73)
    at org.openhab.binding.weather.internal.bus.WeatherBinding.activate(WeatherBinding.java:55)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.eclipse.equinox.internal.ds.model.ServiceComponent.activate(ServiceComponent.java:235)
    at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.activate(ServiceComponentProp.java:146)
    at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.build(ServiceComponentProp.java:345)
    at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponent(InstanceProcess.java:620)

But I don't think we should mix this issues here.

So at least still a missing import...

[kaikreuzer]

But I don't think we should mix this issues here.

Well, THIS issue is about the question if changing the boot delegation might solve most of the regression that b1 brings. The imports and the xml parser are only two examples of such regressions.

[maggu2810]

I talked about mixing because I added a second issue that is raised if the annotation package is added and the boot delegation is not set for the weather binding. The missing imports of the weather binding could be handled in a separate issue.

If the IDE could be instructed to act as the Karaf container (and perhaps as other plain OSGi frameworks) then this will lead to bundles that are already fixed caused by the tests the developer will do.

[maggu2810]

I would assume, the most errors we have now are caused by different behaviour of the standard Equinox launch configuration (in the IDE and as a stand alone product) and the one Karaf is using. IMHO the Karaf one is more strict and so cleaner as it fits more the OSGi spec.

[kaikreuzer]

Right. But you should note that openHAB users do not care about cleanliness according to OSGi spec. We have 195 add-ons for openHAB 1, which have been reviewed, approved and are successfully in use for many years. Most of them worked smoothly on openHAB alpha2 as well and most seem to be broken now. I am already tempted to completely roll back the switch to Karaf, if we cannot quickly solve these issues. As mentioned above, there are further regressions, e.g. ZWave and KNX (both VERY popular among the users) broke for yet unknown reasons... It is really rather a desaster...

[kaikreuzer]

Especially for parts of the OSGi spec that you only notice at runtime and not at compile time, I very much prefer the runtime to be more forgiving instead of throwing execeptions at our users...

[maggu2810]

I get your points (user experience, etc). If there is more interest to get stuff working as to do it in a spec. compliant way, perhaps the rollback should be done as soon as possible. IMHO some people (at least you, davy and me) invest a lot of time and work to move openHAB to Karaf. I learned a lot, so I am happy what I have done, but if a rollback is considered, we should think about if we should still fix bundles or leave it. The broken stuff is very annoying, but as you already stated in another post (or I read it such way), the stuff was already broken before, now (as Karaf is more stict) we see how broken the bundles has been.

[maggu2810]

I have used "broken"... I think you know what I mean. For the normal user, the stuff is broken, if it is not running. For me as a developer and architect, the stuff is broken, if guidelines and rules are not fulfilled (regardless if this hits a runtime exception).

[kaikreuzer]

Well, reverting all we did would indeed only be a very last resort. Besides all the effort that we put into it during the past weeks, we will also mainly lose all the dependency management, for which another solution would have to be found.

That's why I didn't create this issue asking to revert Karaf, but rather see, if we can configure it in a way that it is more forgiving resp. more similar to the behavior of the previous openHAB runtime. If this "reconfiguration" means that the runtime accepts bundles that are not 100% compliant to the OSGi spec, this would imho be ok - after all, 100% OSGi-spec compatibility was never a requirement for openHAB (not talking about ESH here).

[kaikreuzer]

Btw, other problems seem to be due to the different component activation (immediate, order of method calls of ManagedService, etc.) - this is also something that maybe could be addressed.

As I am not so deep in Karaf as you and @dvanherbergen, I hope for your assistance in finding the best way to move forward. Fact is that I am currently drowning in issues where people say that nothing works as before, which renders the beta1 almost completely unusable.

[ANierbeck]

Sorry to see you have so much trouble with now a clean setup. But let me assure you that it'll pay out in the end. For me openhab1 itself had been so a disappointing application from the developer view that I turned away from it due to all those require bundles dependencies. Cause actually I gave it a try to run on karaf about 3 years ago. Right now you are at the point where a lot of people starting with OSGi usually bump into, if it's not a clean setup of bundles, it's a mess and causes a lot of unknown issues. But is the messenger to blame that the setup is rubbish? I don't think so, so let's get on with it and cleanup those bundles that are of an issue. In the end you'll have a much cleaner setup. And if you walk the way of OSGi (100% spec compatible was never a requirement for openHAB) you should really use it or leave it at all, there isn't a way of in between or you end in such a mess, those "old" bundles are bringing you in right now. So I'd suggest rather change the "dev"-environment (eclipse) to actually reflect the runtime setup.

[kaikreuzer]

all those require bundles dependencies

We are using import-package since a long time already. I don't know why you think that not importing javax.net results in a "disappointing application".

Cause actually I gave it a try to run on karaf about 3 years ago.

I remember - it was actually you why I have it in the back of my head that users ask for running openHAB on top of Karaf. But thinking about it now, there weren't too many others...

people starting with OSGi

Note that I am doing OSGi development this way since more than 10 years already. Also note that Eclipse (I would claim this is one of the most wide-spread OSGi application in existence) does NOT do any of the things listed above. Actually, openHAB is completely compatible with the way "pure Equinox" works.

you'll have a much cleaner setup

Do you have any arguments what is the benefit to openHAB besides increasing the chances to win a price for the best OSGi spec compatibility?

you should really use it or leave it at all

Is that also your suggestion to the Eclipse IDE? Could you imagine asking these developers now to refactor their entire codebase, just because setting a bootdelegation flag in another way is a "clean" instead of a "rubbish" setup?

So I'd suggest rather change the "dev"-environment (eclipse) to actually reflect the runtime setup.

I'd love too, but I was also very disappointed to learn that https://karaf.apache.org/index/subprojects/eik.html doesn't seem to be maintained and it is simply not possible to launch a Karaf-based runtime from within the IDE. This is yet another argument against using Karaf, since our previous setup was much easier for all our developers.

Sorry, but somehow I feel this doesn't bring us any closer to a solution of the current (urgent!) problems.

[xsnrg]

I would really hate to see all the advantages that karaf brings be reverted. Is it possible to build two runtimes for a while until karaf gets sorted out, or did the code base change too heavily to support karaf that it really isn't feasible outside of a rollback? The problem with that would be the limited resources would be futher split. The advantage would be OH2 would be usable with current code as before.

A diversion from "clean" to get it working again would not be a big issue either as long as those changes are understood and later worked through.

I am not a core dev though, so my opinion is just that.

[dvanherbergen]

Fact is that I am currently drowning in issues where people say that nothing works as before, which renders the beta1 almost completely unusable.

Maybe we should do some damage control / expectation settings first to reduce the flow of incoming issues. We know that there are some issues with oh2b2, so let's communicate this clearly so that people know what (not) to expect of this beta release. At this point, I would only recommend beta2 for binding developers to check if their bindings work. End users should not use it yet, and if they do want to give it a go, it is probably best that at this time they limit themselves to installing addons using the feature:install (or via the UI's :wink: ), but not using the addons folder.

To me though, it doesn't look like it will take weeks to get the quality where it should be.

And to go slightly off topic, we may need to start thinking about introducing automated functional tests. As the number of bindings keep growing and more developers on/offboard, this could become a long term nightmare otherwise.

[cschneider]

I fully agree that damage control is the best thing to do right now. So I propose to notify people that openhab 2 beta 2 has known issues and rather should not be used by end users.

I got a bit lost in all the messages in this thread. So I try to recap what I understood.

The OSGi spec states that you should have import package statements for all packages outside of java.*. The current bindings sometimes miss such imports. In openhab 1 this still works as many packages are made available by boot delegation. Karaf does not do this so these bindings fail.

Is this correct?

If yes then I wonder why it works in openhab 1. Did you do anything special on the boot delegation to make this work? I will test a bit with the weather binding on openhab1 and 2 to see if there is a good solution for both environments.

I think correcting the modules should not be a big problem. It is just quite a lot of work, especially to test the modules. The bigger problem would be if these modules then do not work on openhab 1 anymore. Is this the case? If yes then I hope it can be fixed by exporting the respective packages from the system package in openhab 1. Does that make sense?

About the general use of karaf in openhab I am a bit concerned about the different build models. Typical karaf applictions do not use tycho. I can imagine that building with tycho and testing on karaf might be a bit difficult. I also can understand that not being able to start openhab2 (karaf) in eclipse is an issue. We are working with the bndtools team to provide a solution for this. The problem though is that this solution will not be based on tycho. It will also take quite some time until bndtools will fully support karaf and maven.

[kaikreuzer]

the feature:install (or via the UI's :wink: ), but not using the addons folder.

Addons is the only option if your binding is not yet included in the official feature (and btw, all bindings that are were previously successfully tested on OH2!)

To me though, it doesn't look like it will take weeks to get the quality where it should be.

To me it does. I have already been working on fixing issues that only occurred on Karaf for the past two months.

So I propose to notify people that openhab 2 beta 2 has known issues and rather should not be used by end users.

Well, this more or less means, taking the beta1 release back and asking people again to use alpha2 instead. Note that there are already many people using it successfully and they implement a lot of features with it.

Is this correct?

Yes, but it is only one of the regressions and probably the one that is easiest to fix (although it means quite some effort and so far I haven't seen a single PR of anyone but me to add such imports...) And I still do not see the benefit of this exercise.

Also think of all the openHAB 1 add-ons that people have developed and not contributed back to the repo. Most of them are likely to be broken then. Maybe people don't even have a dev environment available and only the jar at hand. Why should this not work anymore?

I think correcting the modules should not be a big problem.

It is a big problem. Other issues (that nobody can yet explain) are (not an exhaustive list): https://community.openhab.org/t/zwave-status-under-oh2/1661/72 https://community.openhab.org/t/error-on-weather-binding-when-starting-openhab2/6081 https://github.com/openhab/openhab2-addons/issues/631#issuecomment-172370772 https://github.com/openhab/openhab2-addons/issues/625

Back to my question in this issue: Is there a way to make the Karaf distro more compatible with the openHAB 1 runtime?

[kaikreuzer]

I would really hate to see all the advantages that karaf brings be reverted.

What exactly are these to you?

Is it possible to build two runtimes for a while

We did that for the past 3 months already. Fact is, as long as there is a working runtime (the non-karaf one) nobody really cares about the other one. And it indeed means quite some overhead to maintain that.

[kaikreuzer]

Ok, my suggestion is to enable Equinox compatibility mode for boot delegation as this is what the openHAB 1 runtime has enabled as well - see https://wiki.eclipse.org/Equinox_Boot_Delegation

My arguments for it are exactly the same as on the Eclipse wiki:

The advantage of this approach is it gives us the performance improvements and isolation we want, but it provides a level of compatibility for bundles which expect access to all packages provided by the boot classpath. During the 3.1 development cycle we attempted to disable boot delegation by default. In the end we had to back out of this change because we did not have a good story for backwards compatibility. With the proposed solution we should have much better success.

Submitted https://github.com/openhab/openhab-distro/pull/93

[dvanherbergen]

Sounds good to me.

[maggu2810]

I haven't seen a single PR of anyone but me to add such imports

I think such statements are not fair. I am using Karaf before we started the branch for Karaf support and without looking at the PR I am really sure there are multiple ones for ESH + OH that has been created by myself. I created no further one the last time because 1. I have not much time ATM 2. the bundles that I am using does not throw any error.

IMHO there are much frustration and feelings. We should keep in mind to find a solution to make OH working for the users and improve compatibility. Perhaps we could create another distro that is used by the developers that is not using boot delegation and other workarounds and others for the users (no devs) that contains workarounds etc. We could also create a wiki page that has a matrix for all bundles "works for OH2 compat, works for OH2 strict" etc.

We could also start to look for a better Karaf IDE integration. We could also start to look for stuff we could add to Eclipse IDE, launch config, PDE, ... settings to find errors in the development phase etc.

[kaikreuzer]

I think such statements are not fair.

Sorry, I didn't mean to finger point at anybody here, especially at none of the guys who helped on creating the new distro and VERY especially not at you! I merely intended to point out that the developers of the concerned 1.x add-ons are not really interested in addressing these things quickly (and I do not blame them either, because I can understand it).

[cschneider]

The compatibility mode is pretty ugly but I think it is the best way for now.

Does anyone know if it is possible to let equinox log all classes loaded in this way? This would allow to simply run openhab and then look into the log to see where we can improve the manifests.

[maggu2810]

Created PR to support generation of a manifest using the maven-bundle-plugin (internally the bndtools). Developers could use this to look for missing import statements (if they want). The bootdelegation is no active, but IMHO this is a small change to give support for the lookup.

• openhab/openhab#3833
• openhab/openhab2-addons#632

There has been whitespaces and indentation fixes, too.

[kaikreuzer]

I wonder if we should maybe also add a new launch config with your suggestions from https://github.com/openhab/openhab-distro/issues/85#issuecomment-172309955? So that at least new code that is written comes with the required imports?

[maggu2810]

It would be great if such a launch config exists and is used by more and more developers. BUT I would perhaps wait until:

• the most ESH classes that are used by Gson (sun.misc.Unsafe requirement) offers a no-arg constructor
• the REST API is working using no bootdelegation (it seems it does not work, have to further inspect this one)

[xsnrg]

    I would really hate to see all the advantages that karaf brings be reverted.

What exactly are these to you? 

So far, The console is much easier to use. Tab completion, and log level changing to a running system are very nice. The console log redirected so that the console is usable is also very nice, though that might be something the old container could have done as well. From a user perspective, the ability to manage addons and their dependencies behind the scenes is a big step forward in usability, which I believe ultimately is the goal. There are other things as well, these are just the ones that come to mind with the new changes.

My comment was only to offer encouragement at a time that feels frustrating with the issues presented. I very much appreciate the effort the team is putting into this. The best I can do is to keep testing things and offering feedback unless I can be helpful in other ways. If so, just say.

[kaikreuzer]

Thanks, Jim. I agree that the addon-management and the shell are two excellent features that I would not want to lose either. It is good to know to have forgiving users that are patient when things don't work out as expected.

So just as a general update here: the change in #93 seems to have solved many problems already. Nonetheless, I opened to further issues for stuff that is now broken with the Karaf-based distro:

• https://github.com/openhab/openhab-distro/issues/96
• https://github.com/openhab/openhab-distro/issues/94

I'd welcome very much any idea of what we could do about them!

[kaikreuzer]

It is good to know to have forgiving users that are patient when things don't work out as expected.

But there are also other users around: https://community.openhab.org/t/insteonplm-or-any-working-1-x-binding-in-oh2b1/6414