search

linkedin / cruise-control

Cruise-control is the first of its kind to fully automate the dynamic workload rebalance and self-healing of a Kafka cluster. It provides great value to Kafka users by simplifying the operation of Kafka clusters.
https://github.com/linkedin/cruise-control/tags
BSD 2-Clause "Simplified" License
2.74k stars 585 forks source link

Is there a final jar available with all dependency jars in jfrog #2151

Closed sappusaketh closed 1 month ago

sappusaketh commented 4 months ago

I have downloaded cruise control , core and metrics jar from jfrog now I m trying to start cruise control with ./kafka-cruise-control-start.sh config/cruisecontrol.properties but getting error 

Exception in thread "main" java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
        at com.linkedin.kafka.cruisecontrol.KafkaCruiseControlMain.<clinit>(KafkaCruiseControlMain.java:18)
Caused by: java.lang.ClassNotFoundException: org.slf4j.LoggerFactory
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(Unknown Source)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(Unknown Source)
        at java.base/java.lang.ClassLoader.loadClass(Unknown Source)
        ... 1 more

I think its because dependency jars are missing in class path anyway to download jars from jfrog and start cruise control?

marcelloromani commented 4 months ago

What worked in my experience is performing the build before attempting to start CC with the shell script.

sappusaketh commented 4 months ago

ya currently doing that but trying to see if I can reduce my build time by downloading already built jar

marcelloromani commented 4 months ago

I don't think build time is the issue here. How many times do you need to build CC's jars? Once per version. If you need to re-build CC every time you deploy then you have an issue in your pipeline. Just build CC, store the resulting jars somewhere and consume them.

sappusaketh commented 4 months ago

How many times do you need to build CC's jars? Once per version

Its once per version but My docker file do have other components like CC UI and nginx and if I have to change anything in dockerfile I m building the jars

Other option I considered is building CC in to its own docker image and reference it in my docker file which has other components but that will add additional things to manage in my cicd so planning to see if there is a way I can reduce build times during docker build using existing jars published to jfrog

marcelloromani commented 4 months ago

We solved the issue by having one repository build and publishing the jars. Another repo uses those jars and adds CC UI and other minor bits that we need in our specific use case. In other words, separate building CC and building other stuff.

mhratson commented 1 month ago

Other option I considered is building CC in to its own docker image and reference it in my docker file which has other components but that will add additional things to manage in my cicd so planning to see if there is a way I can reduce build times during docker build using existing jars published to jfrog

Maybe multistage docker build or build caches can help? In any case not a CC issue.