features update lifecycle

[jjongsma]

Removing a repository from an application's feature.xml files should remove it from Karaf, and uninstall/unregister any bundles that no longer exist in active repositories.

Use case: I have a nearly-optimal deployment setup for news, where the feature.xml is automatically generated:

https://nexus.barchart.com/service/local/repositories/releases/content/com/barchart/news/barchart-news-instance-feed-01/1.0.7/barchart-news-instance-feed-01-1.0.7-features.xml

This is referenced by the main news application's feature.xml:

https://github.com/barchart/barchart-configuration/blob/news-service-061/instance/com/barchart/aws/news/feed-01/feature.xml

Then I released a few new bundles and updated the news config to the following:

https://nexus.barchart.com/service/local/repositories/releases/content/com/barchart/news/barchart-news-instance-feed-01/1.0.8/barchart-news-instance-feed-01-1.0.8-features.xml

https://github.com/barchart/barchart-configuration/blob/news-service-062/instance/com/barchart/aws/news/feed-01/feature.xml

The live instance picked up the changes, but then displayed this in the log:

2013-03-25 17:15:41,248 | ERROR | lixDispatchQueue | scr | 115 - org.apache.felix.scr - 1.6.2 | Cannot use configuration pid=barchart.osgi.http.service for bundle mvn:com.barchart.osgi/barchart-osgi-http/1.0.9 because it belongs to bundle mvn:com.barchart.osgi/barchart-osgi-http/1.0.8

It appears the old bundle versions are still hanging around, taking up server PIDs and other resources (port bindings, etc), and I don't know how to get rid of them - I assume this has to be part of the deployer.

(I am unable to verify actual bundles installed on the instance from the console, since SSH is broken in the latest Karaf snapshot)

[Andrei-Pozolotin]

status please

[jjongsma]

Still an issue, ideally if the original reference to the remote repository disappears, that repository and all of its auto-installed bundles should be deactivated/removed (similar to how the current fileinstall method behaves)

[Andrei-Pozolotin]

right, now I remember - greek reference counting in features service implementation. close/postpone/fixnow?

[jjongsma]

I think postpone for now - we still need to define the "ideal scenario". I think my ideal scenario is 4, although 2 is not bad and does not require messing with remote repositories. What are your thoughts? Maybe you should be a customer of your own product by trying to deploy a new server project before coming to a final opinion.

Options:

1) Monolithic features.xml file with versioned name (current)

Pro: works now Con: update kills entire app, changing filename is inconvenient

Changes needed:

• None

2) Multiple smaller features.xml files in config, with versioned names

Pro: update individual chunks without shutting down whole app Con: maintenance headache, need automated process to avoid deployment errors

Changes needed:

• Have "feature" bundles automatically deploy their own features.xml to maven on build/release. This needs to be configurable on a per-project basis since some bundles will not be standalone features.
• Change feature generation to depend on other features when available
• Update barchart-conf / karaf-base-app to also look for features.xml files in a features/ subdirectory in order to maintain an organized config root
• Change "app builds" (i.e. news instances) into jobs that just pull in different released features.xml files from Maven and copy them to the config repo
• Ensure that Karaf loads all local features.xml files before trying to install any features. In the past Karaf has had issues with trying to install a feature that depended on feature from a different features.xml file, because that file hadn't been processed yet.

3) Multiple remote repositories for smaller features, with aggregating features.xml file in config repo with versioned name

Pro: no copying individual XML files into config, update individual bundles without shutting down app Con: maintenance headache, need automated process to avoid deployment errors

Changes needed:

• Fix this bug
• Change feature generation to depend on other features when possible
• Have "feature" bundles automatically deploy their own features.xml to maven on build/release. This needs to be configurable on a per-project basis since some bundles will not be standalone features.
• Automate generation of aggregate features file with remote repository references

4) Single feature.xml with static name, which has a pointer to a remote repository(s) with multiple features defined in it, When the remote repository reference is updated, features that no longer exist are removed, and new features are installed.

Pro: easiest to generate and maintain, update individual bundles without shutting down app Con: none

Changes needed:

• Fix this bug, loading new repositories before removing old ones, so that we can properly detect which features are still defined in new repository
• Change feature generation to depend on other features when possible
• Patch Karaf/fileinstall to remove features that are no longer defined in any repository after repository updates

[Andrei-Pozolotin]

5) use subsystems.

a) there are 3 close things doing the same stuff

• karaf/features
• eclipse/plans
• spring/applications

b) there is a new standard now "Subsystems Service Specification" which is to replace them all
http://blog.osgi.org/2012/06/core-release-5-and-enterprise-release-5.html
http://www.slideshare.net/mfrancis/subsystems-for-those-occasions-where-bundles-are-just-too-small-tom-watson
http://www.youtube.com/watch?v=3uq729vFQEc&feature=youtu.be

c) it is implemented by aries (50 people from IBM + SAP)
http://aries.apache.org/
http://aries.apache.org/community/people.html

d) migration path:

• same approach as karaf
• different xml, different pluigns
• eval if it really works already or wait till karaf makes the change.

[jjongsma]

In that case I'll stay with 1) for now. I've had my fill of shoehorning bleeding-edge technology into our build process for now. Let's wait and see how it plays out.

[Andrei-Pozolotin]

current karaf answer

http://mail-archives.apache.org/mod_mbox/karaf-dev/201304.mbox/%3C51683C57.2050403%40nanthrax.net%3E