search

linkedin / cruise-control

Cruise-control is the first of its kind to fully automate the dynamic workload rebalance and self-healing of a Kafka cluster. It provides great value to Kafka users by simplifying the operation of Kafka clusters.
https://github.com/linkedin/cruise-control/tags
BSD 2-Clause "Simplified" License
2.74k stars 587 forks source link

Cruise Control Cannot Start on Windows #890

Closed mmanna-sapfgl closed 4 years ago

mmanna-sapfgl commented 5 years ago

Cruise control cannot start after downloading from latest release.

  1. Followed instruction from github page to download cruise-control-2.0.59.jar

  2. Did a git init to initialize and gradlew build.

  3. Using command prompt - went to C:/cruise-control-2.0.59 folder (where it's all extracted).

  4. From config folder - changed values of bootstrap.servers and zookeeper.connect.

  5. Ran gradlew jar copyDependentLibs and then kafka-cruise-control-start.sh -jars ./cruise-control/build/libs (Using Git bash).

Result

Error: Could not find or load main class com.linkedin.kafka.cruisecontrol.KafkaCruiseControlMain

I can confirm that under cruise-control/build/libs all jars are created correctly. Also, I ommited the -jars flag and tried again. Same result.

mmanna-sapfgl commented 5 years ago

Hello - could someone respond to this? We cannot start cruise control in our workstations as well as other places without this. I have retried with fresh installation and it's still having the same issue.

efeg commented 5 years ago

@mmanna-sapfgl Looks like the issue is in the following step:

Copy ./cruise-control-metrics-reporter/build/libs/cruise-control-metrics-reporter-A.B.C.jar (Where A.B.C is the version of the Cruise Control) to your Kafka server dependency jar folder. For Apache Kafka, the folder would be core/build/dependant-libs-SCALA_VERSION/

Can you check if your core/build/dependant-libs-SCALA_VERSION/ folder (where SCALA_VERSION depends on your particular Kafka instance) indeed contains the cruise-control-metrics-reporter-A.B.C.jar?

mmanna-sapfgl commented 5 years ago

@mmanna-sapfgl Looks like the issue is in the following step:

Copy ./cruise-control-metrics-reporter/build/libs/cruise-control-metrics-reporter-A.B.C.jar (Where A.B.C is the version of the Cruise Control) to your Kafka server dependency jar folder. For Apache Kafka, the folder would be core/build/dependant-libs-SCALA_VERSION/

Can you check if your core/build/dependant-libs-SCALA_VERSION/ folder (where SCALA_VERSION depends on your particular Kafka instance) indeed contains the cruise-control-metrics-reporter-A.B.C.jar?

Apologies for late reply. I am not sure what you mean here. Because I repeated all the steps as per wiki. And then looked into cruise-control-code/build/ there is no such jar or folder as you said. THis is all I could see inside cruise-control-core:


forDummy@dummyComputer MINGW64 /c/cruise-control-2.0.59/cruise-control-core/build (master)
$ ls -lhR
.:
total 0
drwxr-xr-x 1 forDummy 1049089 0 Sep  1 11:50 classes/
drwxr-xr-x 1 forDummy 1049089 0 Sep  1 11:50 generated/
drwxr-xr-x 1 forDummy 1049089 0 Sep  1 11:50 libs/
drwxr-xr-x 1 forDummy 1049089 0 Sep  1 11:50 resources/
drwxr-xr-x 1 forDummy 1049089 0 Sep  1 11:50 tmp/

./classes:
total 0
drwxr-xr-x 1 forDummy 1049089 0 Sep  1 11:50 java/

./classes/java:
total 0
drwxr-xr-x 1 forDummy 1049089 0 Sep  1 11:50 main/
drwxr-xr-x 1 forDummy 1049089 0 Sep  1 11:50 test/

./classes/java/main:
total 0
drwxr-xr-x 1 forDummy 1049089 0 Sep  1 11:50 com/

./classes/java/main/com:
total 0
drwxr-xr-x 1 forDummy 1049089 0 Sep  1 11:50 linkedin/

./classes/java/main/com/linkedin:
total 4.0K
drwxr-xr-x 1 forDummy 1049089 0 Sep  1 11:50 cruisecontrol/

./classes/java/main/com/linkedin/cruisecontrol:
total 16K
drwxr-xr-x 1 forDummy 1049089    0 Sep  1 11:50 common/
drwxr-xr-x 1 forDummy 1049089    0 Sep  1 11:50 config/
-rw-r--r-- 1 forDummy 1049089 1.2K Sep  1 11:50 CruiseControlUtils.class
drwxr-xr-x 1 forDummy 1049089    0 Sep  1 11:50 detector/
drwxr-xr-x 1 forDummy 1049089    0 Sep  1 11:50 exception/
drwxr-xr-x 1 forDummy 1049089    0 Sep  1 11:50 metricdef/
drwxr-xr-x 1 forDummy 1049089    0 Sep  1 11:50 model/
drwxr-xr-x 1 forDummy 1049089    0 Sep  1 11:50 monitor/
drwxr-xr-x 1 forDummy 1049089    0 Sep  1 11:50 servlet/

./classes/java/main/com/linkedin/cruisecontrol/common:
total 18K
drwxr-xr-x 1 forDummy 1049089    0 Sep  1 11:50 config/
-rw-r--r-- 1 forDummy 1049089  268 Sep  1 11:50 CruiseControlConfigurable.class
-rw-r--r-- 1 forDummy 1049089  525 Sep  1 11:50 Generationed.class
-rw-r--r-- 1 forDummy 1049089 1.9K Sep  1 11:50 LongGenerationed.class
drwxr-xr-x 1 forDummy 1049089    0 Sep  1 11:50 utils/
-rw-r--r-- 1 forDummy 1049089 2.3K Sep  1 11:50 WindowIndexedArrays.class

./classes/java/main/com/linkedin/cruisecontrol/common/config:
total 121K
-rw-r--r-- 1 forDummy 1049089 2.5K Sep  1 11:50 'AbstractConfig$RecordingMap.class'
-rw-r--r-- 1 forDummy 1049089  13K Sep  1 11:50  AbstractConfig.class
-rw-r--r-- 1 forDummy 1049089  829 Sep  1 11:50  Config.class
-rw-r--r-- 1 forDummy 1049089 2.2K Sep  1 11:50 'ConfigDef$1.class'
-rw-r--r-- 1 forDummy 1049089 1.3K Sep  1 11:50 'ConfigDef$2.class'
-rw-r--r-- 1 forDummy 1049089 2.9K Sep  1 11:50 'ConfigDef$3.class'
-rw-r--r-- 1 forDummy 1049089 1.2K Sep  1 11:50 'ConfigDef$4.class'
-rw-r--r-- 1 forDummy 1049089 3.6K Sep  1 11:50 'ConfigDef$ConfigKey.class'
-rw-r--r-- 1 forDummy 1049089 1.3K Sep  1 11:50 'ConfigDef$Importance.class'
-rw-r--r-- 1 forDummy 1049089 1.2K Sep  1 11:50 'ConfigDef$NonEmptyString.class'
-rw-r--r-- 1 forDummy 1049089 2.2K Sep  1 11:50 'ConfigDef$Range.class'
-rw-r--r-- 1 forDummy 1049089  603 Sep  1 11:50 'ConfigDef$Recommender.class'
-rw-r--r-- 1 forDummy 1049089 1.6K Sep  1 11:50 'ConfigDef$Type.class'
-rw-r--r-- 1 forDummy 1049089  313 Sep  1 11:50 'ConfigDef$Validator.class'
-rw-r--r-- 1 forDummy 1049089 2.0K Sep  1 11:50 'ConfigDef$ValidList.class'
-rw-r--r-- 1 forDummy 1049089 2.3K Sep  1 11:50 'ConfigDef$ValidString.class'
-rw-r--r-- 1 forDummy 1049089 1.4K Sep  1 11:50 'ConfigDef$Width.class'
-rw-r--r-- 1 forDummy 1049089  34K Sep  1 11:50  ConfigDef.class
-rw-r--r-- 1 forDummy 1049089 1.3K Sep  1 11:50  ConfigException.class
-rw-r--r-- 1 forDummy 1049089 3.3K Sep  1 11:50  ConfigValue.class
drwxr-xr-x 1 forDummy 1049089    0 Sep  1 11:50  types/

./classes/java/main/com/linkedin/cruisecontrol/common/config/types:
total 4.0K
-rw-r--r-- 1 forDummy 1049089 985 Sep  1 11:50 Password.class

./classes/java/main/com/linkedin/cruisecontrol/common/utils:
total 4.0K
-rw-r--r-- 1 forDummy 1049089 3.3K Sep  1 11:50 Utils.class

./classes/java/main/com/linkedin/cruisecontrol/config:
total 8.0K
-rw-r--r-- 1 forDummy 1049089 6.8K Sep  1 11:50 CruiseControlConfig.class

./classes/java/main/com/linkedin/cruisecontrol/detector:
total 5.0K
-rw-r--r-- 1 forDummy 1049089 242 Sep  1 11:50 Anomaly.class
drwxr-xr-x 1 forDummy 1049089   0 Sep  1 11:50 metricanomaly/

./classes/java/main/com/linkedin/cruisecontrol/detector/metricanomaly:
total 25K
-rw-r--r-- 1 forDummy 1049089  590 Sep  1 11:50 MetricAnomaly.class
-rw-r--r-- 1 forDummy 1049089  773 Sep  1 11:50 MetricAnomalyFinder.class
-rw-r--r-- 1 forDummy 1049089  11K Sep  1 11:50 PercentileMetricAnomalyFinder.class
-rw-r--r-- 1 forDummy 1049089 4.5K Sep  1 11:50 PercentileMetricAnomalyFinderConfig.class

./classes/java/main/com/linkedin/cruisecontrol/exception:
total 5.0K
-rw-r--r-- 1 forDummy 1049089 719 Sep  1 11:50 CruiseControlException.class
-rw-r--r-- 1 forDummy 1049089 492 Sep  1 11:50 NotEnoughValidWindowsException.class

./classes/java/main/com/linkedin/cruisecontrol/metricdef:
total 20K
-rw-r--r-- 1 forDummy 1049089 1.2K Sep  1 11:50 AggregationFunction.class
-rw-r--r-- 1 forDummy 1049089 6.5K Sep  1 11:50 MetricDef.class
-rw-r--r-- 1 forDummy 1049089 1.4K Sep  1 11:50 MetricInfo.class
-rw-r--r-- 1 forDummy 1049089 1.2K Sep  1 11:50 ValueComputingStrategy.class

./classes/java/main/com/linkedin/cruisecontrol/model:
total 1.0K
-rw-r--r-- 1 forDummy 1049089 585 Sep  1 11:50 Entity.class

./classes/java/main/com/linkedin/cruisecontrol/monitor:
total 0
drwxr-xr-x 1 forDummy 1049089 0 Sep  1 11:50 sampling/

./classes/java/main/com/linkedin/cruisecontrol/monitor/sampling:
total 16K
drwxr-xr-x 1 forDummy 1049089    0 Sep  1 11:50 aggregator/
-rw-r--r-- 1 forDummy 1049089 4.8K Sep  1 11:50 MetricSample.class

./classes/java/main/com/linkedin/cruisecontrol/monitor/sampling/aggregator:
total 132K
-rw-r--r-- 1 forDummy 1049089  11K Sep  1 11:50  AggregatedMetricValues.class
-rw-r--r-- 1 forDummy 1049089 1.5K Sep  1 11:50 'AggregationOptions$Granularity.class'
-rw-r--r-- 1 forDummy 1049089 5.3K Sep  1 11:50  AggregationOptions.class
-rw-r--r-- 1 forDummy 1049089 1.4K Sep  1 11:50  Extrapolation.class
-rw-r--r-- 1 forDummy 1049089 3.5K Sep  1 11:50  MetricSampleAggregationResult.class
-rw-r--r-- 1 forDummy 1049089 1.5K Sep  1 11:50 'MetricSampleAggregator$SampleType.class'
-rw-r--r-- 1 forDummy 1049089  24K Sep  1 11:50  MetricSampleAggregator.class
-rw-r--r-- 1 forDummy 1049089 1.8K Sep  1 11:50 'MetricSampleAggregatorState$1.class'
-rw-r--r-- 1 forDummy 1049089  633 Sep  1 11:50 'MetricSampleAggregatorState$MyAtomicLong.class'
-rw-r--r-- 1 forDummy 1049089 9.4K Sep  1 11:50  MetricSampleAggregatorState.class
-rw-r--r-- 1 forDummy 1049089 5.9K Sep  1 11:50  MetricSampleCompleteness.class
-rw-r--r-- 1 forDummy 1049089 4.7K Sep  1 11:50  MetricValues.class
-rw-r--r-- 1 forDummy 1049089  941 Sep  1 11:50 'RawMetricValues$1.class'
-rw-r--r-- 1 forDummy 1049089  15K Sep  1 11:50  RawMetricValues.class
-rw-r--r-- 1 forDummy 1049089 3.6K Sep  1 11:50  ValuesAndExtrapolations.class
-rw-r--r-- 1 forDummy 1049089  11K Sep  1 11:50  WindowState.class

./classes/java/main/com/linkedin/cruisecontrol/servlet:
total 2.0K
-rw-r--r-- 1 forDummy 1049089 209 Sep  1 11:50 EndPoint.class
-rw-r--r-- 1 forDummy 1049089 140 Sep  1 11:50 EndpointType.class
drwxr-xr-x 1 forDummy 1049089   0 Sep  1 11:50 handler/
drwxr-xr-x 1 forDummy 1049089   0 Sep  1 11:50 parameters/
drwxr-xr-x 1 forDummy 1049089   0 Sep  1 11:50 response/

./classes/java/main/com/linkedin/cruisecontrol/servlet/handler:
total 1.0K
-rw-r--r-- 1 forDummy 1049089 440 Sep  1 11:50 Request.class

./classes/java/main/com/linkedin/cruisecontrol/servlet/parameters:
total 1.0K
-rw-r--r-- 1 forDummy 1049089 431 Sep  1 11:50 CruiseControlParameters.class

./classes/java/main/com/linkedin/cruisecontrol/servlet/response:
total 1.0K
-rw-r--r-- 1 forDummy 1049089 524 Sep  1 11:50 CruiseControlResponse.class

./classes/java/test:
total 0
drwxr-xr-x 1 forDummy 1049089 0 Sep  1 11:50 com/

./classes/java/test/com:
total 0
drwxr-xr-x 1 forDummy 1049089 0 Sep  1 11:50 linkedin/

./classes/java/test/com/linkedin:
total 4.0K
drwxr-xr-x 1 forDummy 1049089 0 Sep  1 11:50 cruisecontrol/

./classes/java/test/com/linkedin/cruisecontrol:
total 8.0K
-rw-r--r-- 1 forDummy 1049089 3.3K Sep  1 11:50 CruiseControlUnitTestUtils.class
-rw-r--r-- 1 forDummy 1049089 1.3K Sep  1 11:50 IntegerEntity.class
drwxr-xr-x 1 forDummy 1049089    0 Sep  1 11:50 monitor/

./classes/java/test/com/linkedin/cruisecontrol/monitor:
total 0
drwxr-xr-x 1 forDummy 1049089 0 Sep  1 11:50 sampling/

./classes/java/test/com/linkedin/cruisecontrol/monitor/sampling:
total 4.0K
drwxr-xr-x 1 forDummy 1049089 0 Sep  1 11:50 aggregator/

./classes/java/test/com/linkedin/cruisecontrol/monitor/sampling/aggregator:
total 48K
-rw-r--r-- 1 forDummy 1049089 3.5K Sep  1 11:50  AggregatedMetricValuesTest.class
-rw-r--r-- 1 forDummy 1049089 1.7K Sep  1 11:50 'MetricSampleAggregatorTest$1.class'
-rw-r--r-- 1 forDummy 1049089  20K Sep  1 11:50  MetricSampleAggregatorTest.class
-rw-r--r-- 1 forDummy 1049089  953 Sep  1 11:50 'RawMetricValuesTest$1.class'
-rw-r--r-- 1 forDummy 1049089  16K Sep  1 11:50  RawMetricValuesTest.class

./generated:
total 0
drwxr-xr-x 1 forDummy 1049089 0 Sep  1 11:50 sources/

./generated/sources:
total 0
drwxr-xr-x 1 forDummy 1049089 0 Sep  1 11:50 annotationProcessor/

./generated/sources/annotationProcessor:
total 0
drwxr-xr-x 1 forDummy 1049089 0 Sep  1 11:50 java/

./generated/sources/annotationProcessor/java:
total 0
drwxr-xr-x 1 forDummy 1049089 0 Sep  1 11:50 main/
drwxr-xr-x 1 forDummy 1049089 0 Sep  1 11:50 test/

./generated/sources/annotationProcessor/java/main:
total 0

./generated/sources/annotationProcessor/java/test:
total 0

./libs:
total 108K
-rw-r--r-- 1 forDummy 1049089 106K Sep  1 11:50 cruise-control-core-0.1.10.jar

./resources:
total 0
drwxr-xr-x 1 forDummy 1049089 0 Sep  1 11:50 test/

./resources/test:
total 4.0K
-rw-r--r-- 1 forDummy 1049089 672 Sep  1 11:50 log4j.properties

./tmp:
total 0
drwxr-xr-x 1 forDummy 1049089 0 Sep  1 11:50 compileJava/
drwxr-xr-x 1 forDummy 1049089 0 Sep  1 11:50 compileTestJava/
drwxr-xr-x 1 forDummy 1049089 0 Sep  1 11:50 jar/

./tmp/compileJava:
total 0

./tmp/compileTestJava:
total 0

./tmp/jar:
total 1.0K
-rw-r--r-- 1 forDummy 1049089 25 Sep  1 11:50 MANIFEST.MF

Also, the error says that it cannot find the Main class. Forgive my ignorance, but I didn't understand how that metrics reporter jar is relevant.

efeg commented 5 years ago

Hi @mmanna-sapfgl

...And then looked into cruise-control-code/build/ there is no such jar or folder as you said...

There are 2 issues / misunderstandings regarding:

  1. Finding cruise-control-metrics-reporter-A.B.C.jar
  2. Copying cruise-control-metrics-reporter-A.B.C.jar to the correct location.

For completeness, here is the relevant instructions from Quick Start:

Copy ./cruise-control-metrics-reporter/build/libs/cruise-control-metrics-reporter-A.B.C.jar 
(Where A.B.C is the version of the Cruise Control) to your Kafka server dependency jar folder. 
For Apache Kafka, the folder would be core/build/dependant-libs-SCALA_VERSION/

Finding cruise-control-metrics-reporter-A.B.C.jar

In your main cruise-control directory (i.e. considered as . in the quick start documentation), there are several folders including cruise-control-metrics-reporter. After you run ./gradlew jar in the cruise-control directory, there will be a build folder under cruise-control-metrics-reporter directory. You should find the jar under libs folder within this directory. If these directories are missing, then you have an issue on the previous step in generating the jar.

Copying cruise-control-metrics-reporter-A.B.C.jar to the correct location (from Cruise Control -> to Kafka).

You should copy this jar file from a specific cruise-control directory to a specific Kafka directory. It seems your example above attempts to copy it from a cruise-control directory to the same cruise-control directory, which is not correct. Metrics reporter jar should be running as the pluggable metric reporter at each broker in the cluster, and by copying the jar to the relevant "Kafka" directory, this step ensures that.

mmanna-sapfgl commented 5 years ago

Hi @mmanna-sapfgl

...And then looked into cruise-control-code/build/ there is no such jar or folder as you said...

There are 2 issues / misunderstandings regarding:

  1. Finding cruise-control-metrics-reporter-A.B.C.jar
  2. Copying cruise-control-metrics-reporter-A.B.C.jar to the correct location.

For completeness, here is the relevant instructions from Quick Start:

Copy ./cruise-control-metrics-reporter/build/libs/cruise-control-metrics-reporter-A.B.C.jar 
(Where A.B.C is the version of the Cruise Control) to your Kafka server dependency jar folder. 
For Apache Kafka, the folder would be core/build/dependant-libs-SCALA_VERSION/

Finding cruise-control-metrics-reporter-A.B.C.jar

In your main cruise-control directory (i.e. considered as . in the quick start documentation), there are several folders including cruise-control-metrics-reporter. After you run ./gradlew jar in the cruise-control directory, there will be a build folder under cruise-control-metrics-reporter directory. You should find the jar under libs folder within this directory. If these directories are missing, then you have an issue on the previous step in generating the jar.

Copying cruise-control-metrics-reporter-A.B.C.jar to the correct location (from Cruise Control -> to Kafka).

You should copy this jar file from a specific cruise-control directory to a specific Kafka directory. It seems your example above attempts to copy it from a cruise-control directory to the same cruise-control directory, which is not correct. Metrics reporter jar should be running as the pluggable metric reporter at each broker in the cluster, and by copying the jar to the relevant "Kafka" directory, this step ensures that.

The step mentions that it's only necessary if metrics reporter is used (Default mode for cruise-control). However, I don't see an alternative without it. So it's a deficiency/ambiguity in documentation.

Secondly, this statement

Copy ./cruise-control-metrics-reporter/build/libs/cruise-control-metrics-reporter-A.B.C.jar (Where A.B.C is the version of the Cruise Control) to your Kafka server dependency jar folder. For Apache Kafka, the folder would be core/build/dependant-libs-SCALA_VERSION/

This almost gives an indication that you are referring to kafka source code directory. If you download kafka binary, there is no such folder called core/build/dependent-libs. There are only the following directories

bin config libs logs site-docs

Also, if you echo the final command which calls JAVA with classpath, it's correctly referring to the jar which has the main class. So, once again I am not sure how copying this dependent lib has anything to do with this error message. Perhaps, you can explain this part?

mmanna-sapfgl commented 5 years ago

I have identified the problem. It's rather simple, but tricky to realise right away.

If the script is run truly on Windows, even through MinGW shell, it will fail because the java classpath separator (:) for linux is not the same for Windows (;) as a result, no classpath was added.

The way to reproduce this problem is:

1) Git clone - master branch 2) Do gradlew jar copyDependantLibs 3) Start zk and kafka 4) edit config/cruisecontrol.properties to set zk and broker addresses 5) edit config/capacityJBOD.json to adjust broken and log paths.

Once this is done, it starts on windows. I think the kafka-cruise-control-start.sh script should identify if path separator should be either ":" o ";"

Thanks,

efeg commented 4 years ago

@mmanna-sapfgl Thanks for sharing the steps to reproduce the issue, and glad to hear that it is resolved!

dmarupov commented 4 years ago

Hi, I am also trying to follow the QuickStart instructions and ran into the part where I need to copy the cruise-control-metrics-reporter-A.B.C.jar file to core/build/dependant-libs-SCALA_VERSION/. So my questions is kind of similar. I am using CentOS 7 and my kafka installation has no core directory. Are we to create this core/build/dependant-libs-SCALA_VERSION/ directory? Or are you referning to /kafka/libs? Thank you in advance. I would appreciate some guidance on this.

efeg commented 4 years ago

@dmarupov Thanks for reporting the confusion regarding the destination path.

For Kafka release downloads, please use libs/ folder. core/build/dependant-libs-SCALA_VERSION/ is relevant to Kafka source checkouts. Created https://github.com/linkedin/cruise-control/pull/1319 to clarify this.