third party dependencies, version contraints

[maggu2810]

Currently I try to write a list that contains all the packages and versions contraints that are necessary to run Eclipse SmartHome. This list could be added (some time) to the official documentation (if desired), so e.g. solution providers could look up the dependencies that are necessary to setup a product. So it is not about which third party libraries could be used by a binding developer but which stuff if necessary at runtime.

I started to collect a list of all package imports. This list could contain more packages then necessary because libraries could be bundled, imported but also exported, too. But if the packages are exported, they could be used by binding developers and so the version constraints should be known, too.

That is the current list:

com.eclipsesource.jaxrs.provider.security
com.eclipsesource.jaxrs.publisher
com.google.common.base
com.google.common.collect
com.google.gson
com.google.gson.annotations
com.google.gson.internal
com.google.gson.internal.bind
com.google.gson.reflect
com.google.gson.stream
com.google.inject
com.ibm.icu.text
com.ibm.netrexx.process
com.jayway.jsonpath
com.jcraft.jsch
com.sun.media.jai.codec
com.thoughtworks.xstream
com.thoughtworks.xstream.annotations
com.thoughtworks.xstream.converters
com.thoughtworks.xstream.converters.basic
com.thoughtworks.xstream.converters.reflection
com.thoughtworks.xstream.core
com.thoughtworks.xstream.core.util
com.thoughtworks.xstream.io
com.thoughtworks.xstream.io.naming
com.thoughtworks.xstream.io.xml
com.thoughtworks.xstream.mapper
cx.ath.matthew.debug
cx.ath.matthew.unix
cx.ath.matthew.utils
gnu.io
groovy.json
groovy.lang
groovy.util
groovy.xml
io.swagger.annotations
javax.activation
javax.annotation.security
javax.imageio
javax.inject
javax.jmdns
javax.mail
javax.mail.internet
javax.measure
javax.measure.quantity
javax.measure.spi
javax.media.jai
javax.naming
javax.net
javax.net.ssl
javax.script
javax.security.auth
javax.servlet
javax.servlet.http
javax.sound.sampled
javax.swing
javax.swing.border
javax.ws.rs
javax.ws.rs.container
javax.ws.rs.core
javax.ws.rs.ext
javax.xml.bind
javax.xml.namespace
javax.xml.parsers
javax.xml.stream
javax.xml.transform
javax.xml.transform.sax
javax.xml.transform.stream
javax.xml.xpath
jdepend.textui
jdepend.xmlui
junit.framework
net.bytebuddy.dynamic.loading
net.minidev.json
net.minidev.json.annotate
net.minidev.json.parser
net.minidev.json.reader
net.minidev.json.writer
netrexx.lang
org.apache.bcel.classfile
org.apache.bsf
org.apache.commons.codec.binary
org.apache.commons.collections
org.apache.commons.collections.iterators
org.apache.commons.collections.map
org.apache.commons.exec
org.apache.commons.io
org.apache.commons.lang
org.apache.commons.lang.builder
org.apache.commons.lang.reflect
org.apache.commons.lang.time
org.apache.commons.logging
org.apache.commons.net
org.apache.commons.net.bsd
org.apache.commons.net.ftp
org.apache.commons.net.ntp
org.apache.commons.net.telnet
org.apache.commons.net.util
org.apache.karaf.shell.api.action
org.apache.karaf.shell.api.action.lifecycle
org.apache.karaf.shell.api.console
org.apache.log4j
org.apache.log4j.helpers
org.apache.oro.text.regex
org.apache.regexp
org.apache.tools.ant
org.apache.tools.ant.attribute
org.apache.tools.ant.dispatch
org.apache.tools.ant.filters
org.apache.tools.ant.filters.util
org.apache.tools.ant.helper
org.apache.tools.ant.input
org.apache.tools.ant.launch
org.apache.tools.ant.listener
org.apache.tools.ant.property
org.apache.tools.ant.taskdefs
org.apache.tools.ant.taskdefs.compilers
org.apache.tools.ant.taskdefs.condition
org.apache.tools.ant.taskdefs.email
org.apache.tools.ant.taskdefs.launcher
org.apache.tools.ant.taskdefs.optional
org.apache.tools.ant.taskdefs.optional.depend.constantpool
org.apache.tools.ant.taskdefs.optional.extension
org.apache.tools.ant.taskdefs.optional.extension.resolvers
org.apache.tools.ant.taskdefs.optional.javah
org.apache.tools.ant.taskdefs.optional.jsp
org.apache.tools.ant.taskdefs.optional.jsp.compilers
org.apache.tools.ant.taskdefs.optional.native2ascii
org.apache.tools.ant.taskdefs.optional.net
org.apache.tools.ant.taskdefs.rmic
org.apache.tools.ant.types
org.apache.tools.ant.types.optional.image
org.apache.tools.ant.types.resources
org.apache.tools.ant.types.resources.comparators
org.apache.tools.ant.types.resources.selectors
org.apache.tools.ant.types.selectors
org.apache.tools.ant.types.selectors.modifiedselector
org.apache.tools.ant.types.spi
org.apache.tools.ant.util
org.apache.tools.ant.util.depend
org.apache.tools.ant.util.facade
org.apache.tools.ant.util.java15
org.apache.tools.ant.util.optional
org.apache.tools.ant.util.regexp
org.apache.tools.bzip2
org.apache.tools.mail
org.apache.tools.tar
org.apache.tools.zip
org.apache.xalan.trace
org.apache.xalan.transformer
org.apache.xml.resolver
org.apache.xml.resolver.helpers
org.apache.xml.resolver.tools
org.codehaus.groovy.reflection
org.codehaus.groovy.runtime
org.codehaus.groovy.runtime.callsite
org.codehaus.groovy.runtime.typehandling
org.eclipse.californium.core
org.eclipse.californium.core.coap
org.eclipse.californium.core.network
org.eclipse.californium.core.network.config
org.eclipse.californium.elements
org.eclipse.californium.scandium
org.eclipse.californium.scandium.config
org.eclipse.californium.scandium.dtls.pskstore
org.eclipse.core.runtime
org.eclipse.emf.common.notify
org.eclipse.emf.common.util
org.eclipse.emf.ecore
org.eclipse.emf.ecore.resource
org.eclipse.emf.ecore.util
org.eclipse.jdt.annotation
org.eclipse.jetty.client
org.eclipse.jetty.client.api
org.eclipse.jetty.client.util
org.eclipse.jetty.http
org.eclipse.jetty.io
org.eclipse.jetty.proxy
org.eclipse.jetty.server
org.eclipse.jetty.server.handler
org.eclipse.jetty.servlet
org.eclipse.jetty.util
org.eclipse.jetty.util.component
org.eclipse.jetty.util.ssl
org.eclipse.jetty.util.thread
org.eclipse.lsp4j.jsonrpc
org.eclipse.lsp4j.launch
org.eclipse.lsp4j.services
org.eclipse.osgi.framework.console
org.eclipse.paho.client.mqttv3
org.eclipse.paho.client.mqttv3.logging
org.eclipse.paho.client.mqttv3.persist
org.eclipse.paho.client.mqttv3.util
org.eclipse.xtext
org.eclipse.xtext.common.types.impl
org.eclipse.xtext.conversion
org.eclipse.xtext.ide
org.eclipse.xtext.ide.server
org.eclipse.xtext.nodemodel
org.eclipse.xtext.resource
org.eclipse.xtext.resource.impl
org.eclipse.xtext.util
org.eclipse.xtext.workspace
org.eclipse.xtext.xbase
org.eclipse.xtext.xbase.interpreter
org.eclipse.xtext.xbase.interpreter.impl
org.eclipse.xtext.xbase.lib
org.freedesktop
org.freedesktop.dbus
org.freedesktop.dbus.exceptions
org.glassfish.hk2.utilities.binding
org.glassfish.jersey.internal.inject
org.glassfish.jersey.media.sse
org.glassfish.jersey.server
org.glassfish.jersey.servlet.spi
org.glassfish.jersey.uri.internal
org.hamcrest
org.hamcrest.collection
org.hamcrest.collections
org.hamcrest.core
org.hamcrest.number
org.hamcrest.object
org.joda.time
org.joda.time.base
org.junit
org.junit.matchers
org.junit.rules
org.junit.runner
org.junit.runner.manipulation
org.junit.runner.notification
org.junit.runners
org.junit.runners.model
org.jupnp
org.jupnp.controlpoint
org.jupnp.mock
org.jupnp.model
org.jupnp.model.action
org.jupnp.model.gena
org.jupnp.model.message
org.jupnp.model.message.header
org.jupnp.model.meta
org.jupnp.model.state
org.jupnp.model.types
org.jupnp.registry
org.mapdb
org.mockito
org.mockito.invocation
org.mockito.stubbing
org.mockito.verification
org.objenesis
org.osgi.framework
org.osgi.framework.dto
org.osgi.service.cm
org.osgi.service.component
org.osgi.service.component.runtime
org.osgi.service.component.runtime.dto
org.osgi.service.device
org.osgi.service.event
org.osgi.service.http
org.osgi.service.packageadmin
org.osgi.util.tracker
org.quartz
org.quartz.commonj
org.quartz.core
org.quartz.core.jmx
org.quartz.ee.jmx.jboss
org.quartz.ee.jta
org.quartz.ee.servlet
org.quartz.helpers
org.quartz.impl
org.quartz.impl.calendar
org.quartz.impl.jdbcjobstore
org.quartz.impl.jdbcjobstore.oracle
org.quartz.impl.jdbcjobstore.oracle.weblogic
org.quartz.impl.matchers
org.quartz.impl.triggers
org.quartz.jobs
org.quartz.jobs.ee.ejb
org.quartz.jobs.ee.jms
org.quartz.jobs.ee.jmx
org.quartz.jobs.ee.mail
org.quartz.listeners
org.quartz.management
org.quartz.plugins
org.quartz.plugins.history
org.quartz.plugins.management
org.quartz.plugins.xml
org.quartz.simpl
org.quartz.spi
org.quartz.utils
org.quartz.utils.counter
org.quartz.utils.counter.sampled
org.quartz.utils.weblogic
org.quartz.xml
org.slf4j
org.w3c.dom
org.xml.sax
org.xml.sax.helpers
tec.uom.lib.common
tec.uom.se
tec.uom.se.format
tec.uom.se.function
tec.uom.se.quantity
tec.uom.se.unit

• The minimum OSGi version is part of our documentation.
• Jetty is known (to me): [9.3.15;9.4.0)
• GSON is known (to me): [2.2.4;3.0.0)

For all the other ones, I can lookup which version we are currently using (e.g. the target platform, Karaf features, ...), but I assume this is not the minimum one that is supported (e.g. our target platform does not use the minimum version of OSGi).

@kaikreuzer How to proceed?

[maggu2810]

My current summary (regex - space - version constraint):

# GSON
com.google.gson.* [2.2.4;3.0.0)
# Jetty
org.eclipse.jetty.* [9.3.15;9.4.0)
# OSGi Service Platform Release 4 Core Specification Version 4.2
org.osgi.framework.* 1.5
org.osgi.service.packageadmin.* 1.2
# OSGi Service Platform Release 4 Service Compendium Version 4.2
org.osgi.service.cm.* 1.3
org.osgi.service.component.* 1.1
org.osgi.service.device.* 1.1
org.osgi.service.event.* 1.2
org.osgi.service.http.* 1.2
org.osgi.util.tracker.* 1.4

[kaikreuzer]

But if the packages are exported, they could be used by binding developers and so the version constraints should be known, too.

It would be terrible if binding developers would use all what is available in the target platform. In https://www.eclipse.org/smarthome/documentation/development/bindings/dependencies.html, we tried to define a very small set of packages that should be used. But ok, I see that this question is anyhow not the idea of this effort.

This list could be added (some time) to the official documentation (if desired), so e.g. solution providers could look up the dependencies that are necessary to setup a product.

I do not think that a single list makes sense here - we need it grouped into higher-level features, considering that ESH is pretty modular, i.e. most commercial solutions won't require EMF/Xtext etc. and we thus should not list those as a requirement. The sample-packaging is probably a good starting point for the "minimal requirements" for a solution.

We already have a place in the documentation, where information about 3rd party dependencies is maintained - if I get you right, you would want to move that to a more technical (package versioning) level in order to properly formulate the technical requirements?

[maggu2810]

Currently I am interested in a list that contains all version constraints with respect to all ESH bundles.

As written in the GSON issue some time ago, I don't think it make sense if ESH_bundle_1 and ESH_bundle_2 uses different version requirements for the same package.

So, after this list has been created it is very easy to filter the dependencies as we just need to have a look at the package imports used in the specific manifest. That will be done by a tool.

Currently I am using the list above also to pre-process the ESH bundles automatically to add that versions constraints to the import package statements in the manifest. Using that approach there is one central place that holds the version constraints and the OSGi runtime is able to consider that information for wiring the ESH bundles.

[maggu2810]

The situation improved a lot after dropping the manifest first approach and the migration to Bnd.