strimzi / strimzi-kafka-operator

Apache Kafka® running on Kubernetes
https://strimzi.io/
Apache License 2.0
4.8k stars 1.28k forks source link

[Bug] some cruise control config not working #3551

Closed sangcom15 closed 4 years ago

sangcom15 commented 4 years ago

Describe the bug i was set cruiseControl with strimzi Kafka CRD and specify "anomaly.detection.goals" and "anomaly.detection.interval.ms" in spec.cruiseControl.config but this config is not applied!

To Reproduce

Expected behavior i expected applied that configs

Environment (please complete the following information):

YAML files and logs

apiVersion: kafka.strimzi.io/v1beta1
kind: Kafka
metadata:
  name: central
spec:
  kafka:
    ...
  cruiseControl:
   ...
    config:
      anomaly.notifier.class: com.linkedin.kafka.cruisecontrol.detector.notifier.SlackSelfHealingNotifier
      anomaly.detection.goals: com.linkedin.kafka.cruisecontrol.analyzer.goals.RackAwareGoal,com.linkedin.kafka.cruisecontrol.analyzer.goals.ReplicaCapacityGoal,com.linkedin.kafka.cruisecontrol.analyzer.goals.DiskCapacityGoal,com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkInboundCapacityGoal,com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkOutboundCapacityGoal,com.linkedin.kafka.cruisecontrol.analyzer.goals.CpuCapacityGoal,com.linkedin.kafka.cruisecontrol.analyzer.goals.TopicReplicaDistributionGoal,com.linkedin.kafka.cruisecontrol.analyzer.goals.LeaderReplicaDistributionGoal
      anomaly.detection.interval.ms: 60000
...
...
2020-08-26T06:52:02.124627455Z OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
2020-08-26T06:52:03.322668451Z 2020-08-26 06:52:03,321 INFO KafkaCruiseControlConfig values: 
2020-08-26T06:52:03.322701721Z  add.broker.parameters.class = class com.linkedin.kafka.cruisecontrol.servlet.parameters.AddBrokerParameters
2020-08-26T06:52:03.322716121Z  add.broker.request.class = class com.linkedin.kafka.cruisecontrol.servlet.handler.async.AddBrokerRequest
2020-08-26T06:52:03.322719111Z  admin.parameters.class = class com.linkedin.kafka.cruisecontrol.servlet.parameters.AdminParameters
2020-08-26T06:52:03.322721931Z  admin.request.class = class com.linkedin.kafka.cruisecontrol.servlet.handler.sync.AdminRequest
2020-08-26T06:52:03.322724511Z  allow.capacity.estimation.on.proposal.precompute = true
2020-08-26T06:52:03.322727011Z  anomaly.detection.allow.capacity.estimation = true
2020-08-26T06:52:03.322729491Z  anomaly.detection.goals = [com.linkedin.kafka.cruisecontrol.analyzer.goals.RackAwareGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.ReplicaCapacityGoal, com.linkedin.kafka.cruisecontrol.analyzer.goals.DiskCapacityGoal]
2020-08-26T06:52:03.322733042Z  anomaly.detection.interval.ms = 300000
2020-08-26T06:52:03.322735942Z  anomaly.notifier.class = class com.linkedin.kafka.cruisecontrol.detector.notifier.SlackSelfHealingNotifier
2020-08-26T06:52:03.322738902Z  bootstrap.parameters.class = class com.linkedin.kafka.cruisecontrol.servlet.parameters.BootstrapParameters
2020-08-26T06:52:03.322741492Z  bootstrap.request.class = class com.linkedin.kafka.cruisecontrol.servlet.handler.sync.BootstrapRequest
...
scholzj commented 4 years ago

@kyguy @ppatierno @tomncooper Any ideas?

tomncooper commented 4 years ago

All anaomaly.detection.* configs are blocked. We don't currently support (or expose to the user) any of the Anomaly Detectors features (beyond some metrics). In future we hope to support these features, however only full cluster rebalances are currently supported.

sangcom15 commented 4 years ago

Thank you for answer. If so, it would be nice to specify configs that are not supported in the documentation.

scholzj commented 4 years ago

Not sure if you would think it is sufficient ... but the forbidden options are listed here: https://strimzi.io/docs/operators/latest/full/using.html#type-CruiseControlSpec-reference