connect-jib container will not start

[oneofthemany]

running the docker-compose.yaml returns the following:

Exception in thread "main" java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
        at org.apache.kafka.connect.cli.ConnectDistributedWrapper.<clinit>(ConnectDistributedWrapper.java:20)
Caused by: java.lang.ClassNotFoundException: org.slf4j.LoggerFactory
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(Unknown Source)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(Unknown Source)
        at java.base/java.lang.ClassLoader.loadClass(Unknown Source)
        ... 1 more
Streaming disconnected

Kafka and Zookeeper are both running as expected

:)

[OneCricketeer]

Hi. Thanks for reporting.

Are you running the image from Docker hub, or have you built it locally following the README?

[oneofthemany]

Hi. Thanks for reporting.

Are you running the image from Docker hub, or have you built it locally following the README?

I've used your docker-compose.yaml file and then used podman compose up -d from within the directory of the yaml file. This is what I do usually but if there are steps I have missed it would be appreciated if you could let me know where I'm going wrong.

[OneCricketeer]

I don't use podman, but that shouldn't matter. I was only curious about if you ran make

[oneofthemany]

I don't use podman, but that shouldn't matter. I was only curious about if you ran make

Yeah I prefer podman to docker but it's basically the same and I haven't had any issues with any other containers building

[OneCricketeer]

I wasn't really asking if other containers were the problem. I'm wondering if the main branch is broken, or if the image in Docker Hub is, and it needs re-pushed

[OneCricketeer]

I'll look at it later today

[oneofthemany]

I'll look at it later today

Thanks a lot much appreciated 👍

[OneCricketeer]

I cannot reproduce this with Docker for Mac. Perhaps podman cannot interpret the image format the jib produces.

[oneofthemany]

I cannot reproduce this with Docker for Mac. Perhaps podman cannot interpret the image format the jib produces.

So your build has slf4j within the container?

Are you simply running docker compose up -d?

[OneCricketeer]

It starts fine from the compose file and has valid log output, so yes

[oneofthemany]

It starts fine from the compose file and has valid log output, so yes

And your pulling from dockerio?

I'll give it a go on my Mac to set if it works there and I've got another Linux server that I can chuck docker on as opposed to podman to test also

[OneCricketeer]

I pulled and built fresh. Both worked for Docker for Mac. I didn't try podman.

I spent yesterday upgrading the image and adding arm64 support, just in case people try to run on M1 Mac, for example.

[oneofthemany]

here is a thought - seeing as it seems to think the jar file is missing in the build

what's the location of the jar file in your container?

also can you provide me the jar file so i can map it to the container?

:)

[OneCricketeer]

There are no jars in the container. That's not how Jib works.

The app classpath is located under /app

[oneofthemany]

there are jar files in the app/libs dir - seems one of my mqtt jar files was clashing and I have removed it from app/libs and all is working

curl http://192.168.1.40:9083/connector-plugins [{"class":"org.apache.kafka.connect.file.FileStreamSinkConnector","type":"sink","version":"3.2.3"},{"class":"org.apache.kafka.connect.file.FileStreamSourceConnector","type":"source","version":"3.2.3"},{"class":"org.apache.kafka.connect.mirror.MirrorCheckpointConnector","type":"source","version":"3.2.3"},{"class":"org.apache.kafka.connect.mirror.MirrorHeartbeatConnector","type":"source","version":"3.2.3"},{"class":"org.apache.kafka.connect.mirror.MirrorSourceConnector","type":"source","version":"3.2.3"}]%

[oneofthemany]

I am trying to integrate this:

https://github.com/johanvandevenne/kafka-connect-mqtt

but I am not seeing the MQTT Source and Sink Connectors:

{
    "class": "MQTTSinkConnector",
    "type": "sink",
    "version": "1.0.0"
},
{
    "class": "MQTTSourceConnector",
    "type": "source",
    "version": "1.0.0"
},

Is this something you are aiming at integrating?

[OneCricketeer]

Is this something you are aiming at integrating?

No. Please refer README where I explain this in detail - https://github.com/OneCricketeer/apache-kafka-connect-docker#extending-with-new-connectors

[oneofthemany]

Is this something you are aiming at integrating?

No. Please refer README where I explain this in detail - https://github.com/OneCricketeer/apache-kafka-connect-docker#extending-with-new-connectors

Yeah I had read that, but on the hope that when i pushed a plugin to app/libs that it would work

Its cool I will just go with lenses.io stack as it seems to be comprehensive and has mqtt

Thanks for the support much appreciated

[OneCricketeer]

when i pushed a plugin to app/libs that it would work

It should, but the base image that you're pulling, as you mention from the Docker Compose file doesn't have it.

I tested using schema registry only, as shown here, and those plugins worked fine.

Without seeing your Dockerfile, or how you've edited the compose file, I'm not sure what you missed.

[OneCricketeer]

Going to close this. The linked MQTT connector incorrectly depends on log4j-core rather than using an slf4j bridge. Therefore it conflicts with the classpath of this image.

As stated in the README, there is no guarantee that any connector you choose to install will be compatible with the image.