search

apache / camel-karaf

Apache Camel Karaf support
https://camel.apache.org
Apache License 2.0
28 stars 61 forks source link

camel debezium #546

Open mikadev opened 1 week ago

mikadev commented 1 week ago

Hello maybe it's just me but whit latest Karaf (fresh install) i just install latest camel (4.8.1) when i do:

feature:install camel-debezium-mysql all my ram is eating nothing happen no log just freeze my laptop

mikadev commented 1 week ago

At the end after restarting karaf i just got this one

java.lang.IllegalArgumentException: Unable to find class org.apache.kafka.connect.json.JsonConverter
        at io.debezium.config.Instantiator.getInstanceWithProvidedConstructorType(Instantiator.java:68) ~[?:?]
        at io.debezium.config.Instantiator.getInstance(Instantiator.java:33) ~[?:?]
        at io.debezium.embedded.EmbeddedEngine.<init>(EmbeddedEngine.java:352) ~[?:?]
        at io.debezium.embedded.EmbeddedEngine$EngineBuilder.build(EmbeddedEngine.java:166) ~[?:?]
        at io.debezium.embedded.EmbeddedEngine$EngineBuilder.build(EmbeddedEngine.java:88) ~[?:?]
        at io.debezium.embedded.ConvertingEngineBuilder.build(ConvertingEngineBuilder.java:155) ~[?:?]
        at org.apache.camel.component.debezium.DebeziumConsumer.createDbzEngine(DebeziumConsumer.java:75) ~[?:?]
        at org.apache.camel.component.debezium.DebeziumConsumer.doStart(DebeziumConsumer.java:52) ~[?:?]
        at org.apache.camel.support.service.BaseService.start(BaseService.java:123) ~[?:?]
        at org.apache.camel.support.service.ServiceHelper.startService(ServiceHelper.java:126) ~[?:?]
        at org.apache.camel.impl.engine.AbstractCamelContext.startService(AbstractCamelContext.java:3212) ~[?:?]
        at org.apache.camel.impl.engine.InternalRouteStartupManager.doStartOrResumeRouteConsumers(InternalRouteStartupManager.java:428) ~[?:?]
        at org.apache.camel.impl.engine.InternalRouteStartupManager.doStartRouteConsumers(InternalRouteStartupManager.java:344) ~[?:?]
        at org.apache.camel.impl.engine.InternalRouteStartupManager.safelyStartRouteServices(InternalRouteStartupManager.java:222) ~[?:?]
        at org.apache.camel.impl.engine.InternalRouteStartupManager.doStartOrResumeRoutes(InternalRouteStartupManager.java:132) ~[?:?]
        at org.apache.camel.impl.engine.AbstractCamelContext.doStartCamel(AbstractCamelContext.java:2898) ~[?:?]
        at org.apache.camel.impl.engine.AbstractCamelContext.doStartContext(AbstractCamelContext.java:2528) ~[?:?]
        at org.apache.camel.impl.engine.AbstractCamelContext.doStart(AbstractCamelContext.java:2483) ~[?:?]
        at org.apache.camel.support.service.BaseService.start(BaseService.java:123) ~[?:?]
        at org.apache.camel.impl.engine.AbstractCamelContext.start(AbstractCamelContext.java:2087) ~[?:?]
        at org.apache.camel.impl.DefaultCamelContext.start(DefaultCamelContext.java:211) ~[?:?]
        at org.apache.camel.blueprint.BlueprintCamelContext.start(BlueprintCamelContext.java:234) ~[?:?]
        at org.apache.camel.blueprint.BlueprintCamelContext.maybeStart(BlueprintCamelContext.java:276) ~[?:?]
        at org.apache.camel.blueprint.BlueprintCamelContext.blueprintEvent(BlueprintCamelContext.java:180) ~[?:?]
        at org.apache.aries.blueprint.container.BlueprintEventDispatcher$3.call(BlueprintEventDispatcher.java:197) ~[?:?]
        at org.apache.aries.blueprint.container.BlueprintEventDispatcher$3.call(BlueprintEventDispatcher.java:195) ~[?:?]
        at java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) ~[?:?]
        at java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) ~[?:?]
        at java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
        at java.lang.Thread.run(Thread.java:1583) [?:?]
Caused by: java.lang.ClassNotFoundException: org.apache.kafka.connect.json.JsonConverter from bundle 153 (dt-integ-cdc)
        at org.apache.camel.karaf.core.utils.BundleDelegatingClassLoader.loadClass(BundleDelegatingClassLoader.java:82) ~[?:?]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:526) ~[?:?]
        at io.debezium.config.Instantiator.getInstanceWithProvidedConstructorType(Instantiator.java:63) ~[?:?]
        ... 34 more
Caused by: java.lang.ClassNotFoundException: org.apache.kafka.connect.json.JsonConverter not found by dt-integ-cdc [153]
        at org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1591) ~[?:?]
        at org.apache.felix.framework.BundleWiringImpl.access$300(BundleWiringImpl.java:79) ~[?:?]
        at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(BundleWiringImpl.java:1976) ~[?:?]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:526) ~[?:?]
        at org.apache.felix.framework.Felix.loadBundleClass(Felix.java:2116) ~[?:?]
        at org.apache.felix.framework.BundleImpl.loadClass(BundleImpl.java:986) ~[?:?]
        at org.apache.camel.karaf.core.utils.BundleDelegatingClassLoader.findClass(BundleDelegatingClassLoader.java:48) ~[?:?]
        at org.apache.camel.karaf.core.utils.BundleDelegatingClassLoader.loadClass(BundleDelegatingClassLoader.java:73) ~[?:?]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:526) ~[?:?]
        at io.debezium.config.Instantiator.getInstanceWithProvidedConstructorType(Instantiator.java:63) ~[?:?]
        ... 34 more

and when i do classes |grep org.apache.kafka.connect.json.JsonConverter

135 | org/apache/kafka/connect/json/JsonConverter$1.class | exported: false
135 | org/apache/kafka/connect/json/JsonConverter$2.class | exported: false
135 | org/apache/kafka/connect/json/JsonConverter$3.class | exported: false
135 | org/apache/kafka/connect/json/JsonConverter$4.class | exported: false
135 | org/apache/kafka/connect/json/JsonConverter$5.class | exported: false
135 | org/apache/kafka/connect/json/JsonConverter$JsonToConnectTypeConverter.class | exported: false
135 | org/apache/kafka/connect/json/JsonConverter$LogicalTypeConverter.class | exported: false
135 | org/apache/kafka/connect/json/JsonConverter.class | exported: false
135 | org/apache/kafka/connect/json/JsonConverterConfig.class | exported: false
137 | org/apache/kafka/connect/json/JsonConverter$1.class | exported: false
137 | org/apache/kafka/connect/json/JsonConverter$2.class | exported: false
137 | org/apache/kafka/connect/json/JsonConverter$3.class | exported: false
137 | org/apache/kafka/connect/json/JsonConverter$4.class | exported: false
137 | org/apache/kafka/connect/json/JsonConverter$5.class | exported: false
137 | org/apache/kafka/connect/json/JsonConverter$JsonToConnectTypeConverter.class | exported: false
137 | org/apache/kafka/connect/json/JsonConverter$LogicalTypeConverter.class | exported: false
137 | org/apache/kafka/connect/json/JsonConverter.class | exported: false
137 | org/apache/kafka/connect/json/JsonConverterConfig.class | exported: false
139 | org/apache/kafka/connect/json/JsonConverter$1.class | exported: true
139 | org/apache/kafka/connect/json/JsonConverter$2.class | exported: true
139 | org/apache/kafka/connect/json/JsonConverter$3.class | exported: true
139 | org/apache/kafka/connect/json/JsonConverter$4.class | exported: true
139 | org/apache/kafka/connect/json/JsonConverter$5.class | exported: true
139 | org/apache/kafka/connect/json/JsonConverter$JsonToConnectTypeConverter.class | exported: true
139 | org/apache/kafka/connect/json/JsonConverter$LogicalTypeConverter.class | exported: true
139 | org/apache/kafka/connect/json/JsonConverter.class | exported: true
139 | org/apache/kafka/connect/json/JsonConverterConfig.class | exported: true
144 | org/apache/kafka/connect/json/JsonConverter$1.class | exported: false
144 | org/apache/kafka/connect/json/JsonConverter$2.class | exported: false
144 | org/apache/kafka/connect/json/JsonConverter$3.class | exported: false
144 | org/apache/kafka/connect/json/JsonConverter$4.class | exported: false
144 | org/apache/kafka/connect/json/JsonConverter$5.class | exported: false
144 | org/apache/kafka/connect/json/JsonConverter$JsonToConnectTypeConverter.class | exported: false
144 | org/apache/kafka/connect/json/JsonConverter$LogicalTypeConverter.class | exported: false
144 | org/apache/kafka/connect/json/JsonConverter.class | exported: false
144 | org/apache/kafka/connect/json/JsonConverterConfig.class | exported: false
151 | org/apache/kafka/connect/json/JsonConverter$1.class | exported: false
151 | org/apache/kafka/connect/json/JsonConverter$2.class | exported: false
151 | org/apache/kafka/connect/json/JsonConverter$3.class | exported: false
151 | org/apache/kafka/connect/json/JsonConverter$4.class | exported: false
151 | org/apache/kafka/connect/json/JsonConverter$5.class | exported: false
151 | org/apache/kafka/connect/json/JsonConverter$JsonToConnectTypeConverter.class | exported: false
151 | org/apache/kafka/connect/json/JsonConverter$LogicalTypeConverter.class | exported: false
151 | org/apache/kafka/connect/json/JsonConverter.class | exported: false
151 | org/apache/kafka/connect/json/JsonConverterConfig.class | exported: false
jbonofre commented 1 week ago

@mikadev @essobedo pushed fixes about camel-debezium for camel-karaf 4.8.1 and the itests worked. Can you please provide the list of features alreeady install when you install camel-debezium ? Do you import kafka packages in your dt-integ-cdc bundle ?

mikadev commented 1 week ago

@jbonofre image

 <build>
        <resources>
            <resource>
                <directory>src/main/resources</directory>
                <filtering>true</filtering>
            </resource>
        </resources>
        <plugins>
            <plugin>
                <groupId>org.apache.felix</groupId>
                <artifactId>maven-bundle-plugin</artifactId>
                <version>5.1.9</version>
                <extensions>true</extensions>
                <configuration>
                    <instructions>
                        <Bundle-SymbolicName>${project.artifactId}</Bundle-SymbolicName>
                        <Bundle-Version>${project.version}</Bundle-Version>
                        <Import-Package>*</Import-Package>
                        <Export-Package/>
                    </instructions>
                </configuration>
            </plugin>
        </plugins>
    </build>

    <dependencies>
        <dependency>
            <groupId>org.apache.camel</groupId>
            <artifactId>camel-debezium-mysql</artifactId>
            <version>${camel.version}</version>
            <scope>provided</scope>
        </dependency>
  </dependencies>
essobedo commented 1 week ago

We can try to improve it a bit more, but it should not be too hard to fix it on your side by adding a dynamic import of type org.apache.kafka.connect*.

As a reminder, it is an open-source project, so feel free to propose PRs to improve it, we will be happy to review them

jbonofre commented 1 week ago

@mikadev when you use * for Import-Package, bnd will actually import only the clearly state packages, not necessarily the transitive. In your case, check the MANIFEST to see if kafka packages are there and if not (as I guess), "force" the import by defining the FQN. Something like:

<Import-Package>
   org.apache.kafka.connect,
   org.apache.kafka,
   *
</Import-Package>
mikadev commented 1 week ago

Many thanks for the help :) but not work, still the same

mikadev commented 5 days ago

Hello even by adding Import i have the same error any idea ?