pgjdbc karaf feature file is broken starting with 42.2.13

[steinarb]

I'm submitting a ...

• [X ] bug report
• [ ] feature request

Describe the issue Installing the feature creates a stack trace and terminates ongoing ssh connections.

Sometimes ssh is completely broken and a karaf restart is necessary to get it working again

Driver Version?

42.2.13 through 42.2.16

42.2.12 is the last one that works.

Java Version?

openjdk 11.0.8

OS Version?

debian 10.5 "buster"

PostgreSQL Version?

12.4

To Reproduce Steps to reproduce the behaviour:

1. download the latest karaf and unpack and start it
2. From the command line add the postgresql feature repository and install the feature

   feature:repo-add mvn:org.postgresql/postgresql/42.2.13/xml/features
   feature:install postgresql

After a little while the following stack trace appears:

 org.apache.karaf.deployer.features [org.apache.karaf.deployer.features.osgi.Activator] ERROR : Invalid BundleContext.
java.lang.IllegalStateException: Invalid BundleContext.
        at org.apache.felix.framework.BundleContextImpl.checkValidity(BundleContextImpl.java:511)
        at org.apache.felix.framework.BundleContextImpl.addBundleListener(BundleContextImpl.java:211)
        at org.apache.karaf.deployer.features.FeatureDeploymentListener.init(FeatureDeploymentListener.java:89)
        at org.apache.karaf.deployer.features.osgi.Activator$DeploymentFinishedListener.deploymentEvent(Activator.java:86)
        at org.apache.karaf.features.internal.service.FeaturesServiceImpl.callListeners(FeaturesServiceImpl.java:321)
        at org.apache.karaf.features.internal.service.Deployer.deploy(Deployer.java:1068)
        at org.apache.karaf.features.internal.service.FeaturesServiceImpl.doProvision(FeaturesServiceImpl.java:1062)
        at org.apache.karaf.features.internal.service.FeaturesServiceImpl.lambda$doProvisionInThread$13(FeaturesServiceImpl.java:998)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)

Expected behaviour A clear and concise description of what you expected to happen.

The feature should install without any error message or ssh breakage.

And what actually happens

If installing directly from the console started with karaf the above stack trace appears.

If installing from an ssh session the ssh session is torn down and may occasionally require a karaf restart before ssh starts working again.

Logs If possible PostgreSQL logs surrounding the occurrence of the issue Additionally logs from the driver can be obtained adding

loggerLevel=TRACE&loggerFile=pgjdbc-trace.log 

to the connection string

[steinarb]

The working 4.2.12 karaf feature file looks like this

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<features xmlns="http://karaf.apache.org/xmlns/features/v1.5.0" name="postgresql">
    <feature name="postgresql" description="PostgreSQL JDBC driver karaf feature" version="42.2.12">
        <details>Java JDBC 4.2 (JRE 8+) driver for PostgreSQL database</details>
        <feature>transaction-api</feature>
        <feature prerequisite="true" dependency="false">wrap</feature>
        <bundle start-level="80">mvn:org.postgresql/postgresql/42.2.12</bundle>
        <bundle start-level="80">wrap:mvn:com.ongres.scram/client/2.1</bundle>
        <bundle start-level="80">wrap:mvn:com.github.waffle/waffle-jna/1.9.1</bundle>
    </feature>
</features>

The broken 4.2.13 karaf feature file looks like this

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<features xmlns="http://karaf.apache.org/xmlns/features/v1.5.0" name="postgresql">
  <feature name="postgresql" version="42.2.13" description="PostgreSQL JDBC driver karaf feature">
    <details>Java JDBC 4.2 (JRE 8+) driver for PostgreSQL database</details>
    <feature>transaction-api</feature>
    <bundle>mvn:org.osgi/org.osgi.core/4.3.1</bundle>
    <bundle>mvn:org.osgi/org.osgi.enterprise/4.2.0</bundle>
    <bundle>mvn:org.postgresql/postgresql/42.2.13</bundle>
  </feature>
</features>

Some observations:

1. The new feature is loading old versions of osgi.core and osgi.enterprise into a runtime already containing newer versions of them (osgi 6.0.0 for karaf 4.2x* and osgi 7 for karaf 4.3.x). You shouldn't do this
2. The waffle-jna and scram client that used to be runtime dependencies are not part of the new feature (and since the new feature loads after a fashion, it may be that they are no longer needed...?)

So a minimum fix would be to get rid of loading the old osgi bundles.

(I provided the original karaf feature built with maven, i.e. the one used up to and including 4.2.12, But I have no knowledge of gradle so I can't provide a PR for a fix here)

[steinarb]

I did some experiments at the karaf command line, and the runtime dependencies provided by the built-in karaf feature transaction-api are sufficient, i.e. lose the osgi bundles from the feature and you're good:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<features xmlns="http://karaf.apache.org/xmlns/features/v1.5.0" name="postgresql">
  <feature name="postgresql" version="42.2.13" description="PostgreSQL JDBC driver karaf feature">
    <details>Java JDBC 4.2 (JRE 8+) driver for PostgreSQL database</details>
    <feature>transaction-api</feature>
    <bundle>mvn:org.postgresql/postgresql/42.2.16</bundle>
  </feature>
</features>

Here are the experiments I did from the karaf 4.2.9 console:

karaf@root()> bundle:install mvn:org.postgresql/postgresql/42.2.16
Bundle ID: 48
karaf@root()> bundle:list
START LEVEL 100 , List Threshold: 50
ID │ State     │ Lvl │ Version         │ Name
───┼───────────┼─────┼─────────────────┼────────────────────────────────────────
24 │ Active    │  80 │ 4.2.10.SNAPSHOT │ Apache Karaf :: OSGi Services :: Event
48 │ Installed │  80 │ 42.2.16         │ PostgreSQL JDBC Driver
karaf@root()> bundle:start 48
Error executing command: Error executing command on bundles:
        Error starting bundle 48: Unable to resolve org.postgresql.jdbc [48](R 48.0): missing requirement [org.postgresql.jdbc [48](R 48.0)] osgi.wiring.package; (osgi.wiring.package=javax.transaction.xa) Unresolved requirements: [[org.postgresql.jdbc [48](R 48.0)] osgi.wiring.package; (osgi.wiring.package=javax.transaction.xa)]
karaf@root()> feature:install transaction-api
karaf@root()> bundle:start 48
karaf@root()> bundle:list
START LEVEL 100 , List Threshold: 50
ID │ State  │ Lvl │ Version         │ Name
───┼────────┼─────┼─────────────────┼────────────────────────────────────────────
24 │ Active │  80 │ 4.2.10.SNAPSHOT │ Apache Karaf :: OSGi Services :: Event
48 │ Active │  80 │ 42.2.16         │ PostgreSQL JDBC Driver
49 │ Active │  80 │ 3.0.0           │ Expression Language 3.0 API
50 │ Active │  80 │ 1.2.0           │ CDI APIs
51 │ Active │  80 │ 1.2             │ javax.interceptor API
52 │ Active │  80 │ 1.2             │ javax.transaction API
53 │ Active │  80 │ 1.0.0.2         │ Apache ServiceMix :: Bundles :: javax.inject
54 │ Active │  80 │ 0.4.4           │ pax-transx-tm-api
karaf@root()>

(the rest of the bundles here are pulled in and started by the feature transaction-api)

[steinarb]

Note: If you need those osgi.bundles for the build, and whatever creates the OSGi MANIFEST.MF in gradle, is BND, make sure the osgi bundle have the gradle equivalent of maven "provided" dependencies.

[davecramer]

Is there some way we can get this into our CI pipeline ?

[steinarb]

With maven and java 8: yes (create a pax exam integration test that load the feature in karaf. If the feature loads without issues it is good)

With maven and Java 11: maybe, but I basically gave up on getting my own integration tests working after moving to java 11. It may be that the situation has improved since I last tried.

With gradle: unknown (the pgjdbc project is the first time I've tried to use gradle and the building pgjdbc failed on my dev machine)

But note that getting a working feature again is fairly easy: just remove the start of the old osgi bundles (see the code example in an earlier comment)

[davecramer]

if you can provide me with a travis config that starts it under java 8 with maven I will take it from there.

[vlsi]

For Karaf, I would suggest going with end-to-end test case. I tried pax-exam once, and it seems to be complicated to setup like we have for Scala integration testing: https://github.com/pgjdbc/pgjdbc/tree/master/test-anorm-sbt

I think we would be just fine if we install pgjdbc to the local maven repository, and a test that launches Karaf and verifies if the thing works or not.

[davecramer]

Seems simple enough, easy enough to use maven for the testing as well

[steinarb]

if you can provide me with a travis config that starts it under java 8 with maven I will take it from there.

You need a little more than travis config. You need a JUnit test as well.

Here is an integration test that loads the postgresql karaf feature and verifies that it loads: https://github.com/steinarb/pgjdbc-karaf-ci

(The downside of this test is that it doesn't reveal the error I'm reporting. Bump the pgjdbc version to 42.2.16 and the test runs fine... you'll have to trust me on that loading an old version of OSGi inside an OSGi container is something that shouldn't be done and will have unexpected side effects)

[davecramer]

Do you know how to get Karaf to use a local maven repo?

[steinarb]

Dave Cramer notifications@github.com:

Do you know how to get Karaf to use a local maven repo?

By default karaf uses the local maven repo of the user karaf is started as.

So if the integration test was part of a reactor build, you could e.g. replace the explicit pgjdbc.version in the pom with project.version and start the feature of the current snapshot.

Or you could remove the @Configuration stuff in the test and instead do this: addFeaturesRepository("mvn:org.postgresql/postgresql/LATEST/xml/features"); installAndAssertFeature("postgresql");

NB! Not tested! But this should try installing the most recent snapshot from the local repo.

[davecramer]

Fixed in 42.2.17