Analyze and improve startup time and resource usage

[kaikreuzer]

When trying the runtime on a RasPi 2, it took a couple of minutes to start up, while the "old" distribution takes only around 20 seconds on the same hardware. I assume that there might be still something wrong regarding trying to check external repositories or something. Otherwise I would not know why it could be so much slower.

[maggu2810]

So I think we have to use something in the Eclipse SmartHome framework that does not depend on Karaf and does (hopefully) not depend on start levels.

[dvanherbergen]

Would it be an option that every bundleX register (after successful start) a service bundleXready or something similar and stop that dummy service if it is stopped. Another bundle is using a service tracker to track that services and if all are ready it signals an system completely started (through event or another service)?

That would require knowing up front how many bundles will be loaded...

[kgoderis]

@kgoderis You are well aware that @maggu2810 just mentioned in #514 (comment) that he is actually more rigid than I am when it comes to abolishing start level order of bundles. So no, it might not the best time to come up with that ;-)

Maybe we need even more/better start levels to make the startup more deterministic? behold I am not a Karaf expert at all, I am just suggesting the opportunity ;-)

[kgoderis]

@kgoderis to really eliminate funky stuff startup issues, I believe you would also need to block event publishing during startup. This could be a possible solution:

1. Both the rule engine and event publisher startup in an inactive mode
2. In the core bundle, the system startup level is polled until the system is fully started and then emits a SystemStartEvent.
3. The event publisher picks up the event and starts passing through events. In turn it also emits an BusStartedEvent
4. The rule engine picks up the BusStartedEvent and kicks off the startup rules. With this approach, the start level of bundles is irrelevant.

+1

This could be generalised for every core element. e.g. Item parsing and so forth. Also, we should make sure that any thread that deals with parsing or alike is not exiting ungracefully (I think it is still the case, making debugging troublesome for one)

[maggu2810]

Have another idea. Will check it tomorrow, have to drink beer now ;)

[dvanherbergen]

This could be generalised for every core element. e.g. Item parsing and so forth.

Not only that, but if we think long term, I believe this approach could also work in a distributed setup. So that your rule would not start until the master and all slave instances have started.

@maggu2810 enjoy!

[ANierbeck]

@maggu2810 ... I didn't suggest it :-) I wasn't even aware of that project.

All others, never the less one should really get rid of thinking start-levels is a good idea to manage your application, rather make sure you depend on certain services instead.

regards, Achim

2015-12-17 16:33 GMT+01:00 dvanherbergen notifications@github.com:

This could be generalised for every core element. e.g. Item parsing and so forth.

Not only that, but if we think long term, I believe this approach could also work in a distributed setup. So that your rule would not start until the master and all slave instances have started.

@maggu2810 https://github.com/maggu2810 enjoy!

— Reply to this email directly or view it on GitHub https://github.com/openhab/openhab2/issues/514#issuecomment-165484987.

Apache Member Apache Karaf http://karaf.apache.org/ Committer & PMC OPS4J Pax Web http://wiki.ops4j.org/display/paxweb/Pax+Web/ Committer & Project Lead blog http://notizblog.nierbeck.de/ Co-Author of Apache Karaf Cookbook http://bit.ly/1ps9rkS

Software Architect / Project Manager / Scrum Master

[kaikreuzer]

@kaikreuzer will smarthome also move to karaf?

Eclipse SmartHome will stay a set of OSGi bundles to be picked up. So no, Karaf etc. won't be introduced there.

But what we should do is to design these bundles in a way that solutions like openHAB can define how they want to handle the system lifecycle. As @ANierbeck suggests, in ESH, all we want to make sure is that these bundles are good OSGi citizens and respect the dynamic features of OSGi - thus they should clearly declare their dependencies through services and should cope with any situation where services come and go.

For our concrete problem at hand, we are actually only talking about the right moment to launch the startup rules, right? So in ESH, we could simply remove the logic that says when is "startup" (at the moment when that rule engine bundle is started, which is anyhow a bad idea) and let that be triggered from external, e.g. a bundle that depends on the RuleEngine and calls runStartupRules() on it. For openHAB 2, we could easily implement this using the BootFinished service, don't you think?

Btw, I didn't have a beer tonight, so maybe @maggu2810's idea is better after all ;-)

[maggu2810]

@kaikreuzer Could we move the start / stop code of the bundle activators of the org.eclipse.smarthome.model.*.runtime classes to DS implementations (immediate=true)? So openHAB could create another DS (e.g. BootupDone) that reference the services it needs for startup (cardinality=1..1, policy=static) and trigger runStartupRules in that activate method?

[maggu2810]

So in ESH, we could simply remove the logic that says when is "startup"

As a collection of bundle that are no product themselves we should remove that.

[kaikreuzer]

Could we move the start / stop code of the bundle activators of the org.eclipse.smarthome.model.*.runtime classes to DS implementations (immediate=true)?

Possibly yes, but I can imagine that we run into all kinds of Xtext initialization problems - getting Xtext with its dependencies to start up correctly has always been a challenge... But we can of course have a try.

[maggu2810]

Changes are prepared, will test it later (other stuff to do)... https://github.com/eclipse/smarthome/compare/master...maggu2810:ds-model-runtime?expand=1

[kgoderis]

I am a complete noob when it comes to Karaf (side note: maybe I am not alone but I do not fully grasp what you try to do with Karaf in the first place), but based on what I understand, could a solution not lie in a clever usage of "system" Events on the ESH bus? At least some bindings - if they need to - or other building blocks from the core can figure out when to do stuff, e.g. use RuleEngineStartedEvent, ItemsLoadedEvent and so forth... Something that is maybe not tied to the deep inner workings of the OSGi / Karaf .... And maybe it will make the startup of the whole runtime more deterministic, because, if not, we will end up with a lot of frustrating folks not understanding why their stuff is not working anymore when they go from OH1 to OH2

[dvanherbergen]

So openHAB could create another DS (e.g. BootupDone) that reference the services it needs for startup (cardinality=1..1, policy=static) and trigger runStartupRules in that activate method?

Why create another DS if you can use the karaf built-in BootFinished service? I believe it would be better to use BootFinished instead. This way, you know all bundles will have been loaded. If you create BootupDone, how will you know that all the bindings which are needed to execute the result of the startup rules will be loaded?

based on what I understand, could a solution not lie in a clever usage of "system" Events on the ESH bus?

These are not mutually exclusive. It would be possible to use the BootFinished service to then trigger a SystemStartedEvent.

[kaikreuzer]

Why create another DS if you can use the karaf built-in BootFinished service?

I guess the idea is to have the BootFinished service to depend on BootFinished, RuleEngine, etc. - so it will be activated as soon as all these services are in place.

[dvanherbergen]

Ah, OK. So BootupDone would depend upon BootFinished, so that it activates upon receiving that service. And then it could send the SystemStartedEvent ;-)

[maggu2810]

I have create a PR for Eclipse SmartHome, so we could test if this goes in the right direction. https://github.com/eclipse/smarthome/pull/757

[kaikreuzer]

And for the time being I have created https://github.com/openhab/openhab2/pull/565 as a quick fix :-)

[kaikreuzer]

Closing this as I started follow-up discussion in https://github.com/openhab/openhab-distro/issues/10