strimzi / strimzi-kafka-operator

Apache Kafka® running on Kubernetes
https://strimzi.io/
Apache License 2.0
4.78k stars 1.28k forks source link

[Question] Issues starting kafka operator using helm (helm v2) chart #3965

Closed psamagal closed 3 years ago

psamagal commented 3 years ago

I did try to do a deploy to a K8S cluster (v1.15) of the latest strimzi kafka operator using the Helm (v2) Chart. The version of the chart is strimzi/strimzi-kafka-operator 0.20.0

But when strimzi-cluster-operator pod starts i obtain the following error, no other config changes are being applied apart from default values of the chart

+ export MALLOC_ARENA_MAX=2
+ MALLOC_ARENA_MAX=2
+ JAVA_OPTS=' -Dlog4j2.configurationFile=file:/opt/strimzi/custom-config/log4j2.properties -Dvertx.cacheDirBase=/tmp -Djava.security.egd=file:/dev/./urandom'
++ get_gc_opts
++ '[' '' == true ']'
++ echo ''
+ JAVA_OPTS=' -Dlog4j2.configurationFile=file:/opt/strimzi/custom-config/log4j2.properties -Dvertx.cacheDirBase=/tmp -Djava.security.egd=file:/dev/./urandom '
++ java -version
++ sed -E -n 's/.* version "([0-9]*).*$/\1/p'
+ JAVA_MAJOR_VERSION=11
+ '[' 11 -ge 9 ']'
+ JAVA_OPTS=' -Dlog4j2.configurationFile=file:/opt/strimzi/custom-config/log4j2.properties -Dvertx.cacheDirBase=/tmp -Djava.security.egd=file:/dev/./urandom  --illegal-access=deny'
+ exec /usr/bin/tini -w -e 143 -- java -Dlog4j2.configurationFile=file:/opt/strimzi/custom-config/log4j2.properties -Dvertx.cacheDirBase=/tmp -Djava.security.egd=file:/dev/./urandom --illegal-access=deny -classpath lib/io.strimzi.cluster-operator-0.20.0.jar:lib/com.fasterxml.jackson.core.jackson-core-2.10.4.jar:lib/io.prometheus.simpleclient_common-0.7.0.jar:lib/io.vertx.vertx-micrometer-metrics-3.9.1.jar:lib/io.fabric8.openshift-model-operator-4.12.0.jar:lib/org.apache.zookeeper.zookeeper-3.5.8.jar:lib/org.apache.logging.log4j.log4j-api-2.13.3.jar:lib/com.zaxxer.HikariCP-java7-2.4.13.jar:lib/org.keycloak.keycloak-core-11.0.2.jar:lib/io.fabric8.kubernetes-model-apps-4.12.0.jar:lib/org.bouncycastle.bcprov-jdk15on-1.60.jar:lib/com.squareup.okio.okio-1.15.0.jar:lib/io.fabric8.kubernetes-model-storageclass-4.12.0.jar:lib/org.keycloak.keycloak-common-11.0.2.jar:lib/io.netty.netty-codec-http2-4.1.50.Final.jar:lib/io.fabric8.kubernetes-model-coordination-4.12.0.jar:lib/io.fabric8.kubernetes-model-settings-4.12.0.jar:lib/io.strimzi.kafka-oauth-client-0.6.1.jar:lib/org.xerial.snappy.snappy-java-1.1.7.3.jar:lib/org.hdrhistogram.HdrHistogram-2.1.11.jar:lib/io.prometheus.simpleclient-0.7.0.jar:lib/com.fasterxml.jackson.module.jackson-module-jaxb-annotations-2.10.4.jar:lib/com.sun.activation.jakarta.activation-1.2.1.jar:lib/io.fabric8.kubernetes-model-events-4.12.0.jar:lib/com.squareup.okhttp3.logging-interceptor-3.12.12.jar:lib/javax.xml.bind.jaxb-api-2.3.0.jar:lib/org.apache.yetus.audience-annotations-0.5.0.jar:lib/io.fabric8.kubernetes-model-discovery-4.12.0.jar:lib/javax.annotation.javax.annotation-api-1.3.2.jar:lib/io.fabric8.kubernetes-model-apiextensions-4.12.0.jar:lib/io.fabric8.kubernetes-client-4.12.0.jar:lib/io.strimzi.api-0.20.0.jar:lib/io.netty.netty-handler-proxy-4.1.50.Final.jar:lib/io.netty.netty-codec-dns-4.1.50.Final.jar:lib/io.micrometer.micrometer-core-1.3.1.jar:lib/io.strimzi.certificate-manager-0.20.0.jar:lib/io.fabric8.kubernetes-model-certificates-4.12.0.jar:lib/io.fabric8.openshift-model-monitoring-4.12.0.jar:lib/io.strimzi.kafka-oauth-common-0.6.1.jar:lib/jakarta.activation.jakarta.activation-api-1.2.1.jar:lib/io.netty.netty-transport-native-epoll-4.1.50.Final.jar:lib/io.netty.netty-transport-4.1.50.Final.jar:lib/io.fabric8.kubernetes-model-metrics-4.12.0.jar:lib/com.github.mifmif.generex-1.0.2.jar:lib/io.fabric8.kubernetes-model-common-4.12.0.jar:lib/io.netty.netty-common-4.1.50.Final.jar:lib/io.strimzi.kafka-oauth-server-0.6.1.jar:lib/io.netty.netty-transport-native-unix-common-4.1.50.Final.jar:lib/io.vertx.vertx-core-3.9.1.jar:lib/com.github.luben.zstd-jni-1.4.4-7.jar:lib/io.netty.netty-resolver-dns-4.1.50.Final.jar:lib/org.yaml.snakeyaml-1.26.jar:lib/io.fabric8.kubernetes-model-core-4.12.0.jar:lib/io.fabric8.kubernetes-model-networking-4.12.0.jar:lib/io.strimzi.operator-common-0.20.0.jar:lib/io.fabric8.openshift-model-4.12.0.jar:lib/io.netty.netty-resolver-4.1.50.Final.jar:lib/org.lz4.lz4-java-1.7.1.jar:lib/org.apache.kafka.kafka-clients-2.6.0.jar:lib/io.fabric8.openshift-model-operatorhub-4.12.0.jar:lib/org.apache.logging.log4j.log4j-slf4j-impl-2.13.3.jar:lib/dk.brics.automaton.automaton-1.11-8.jar:lib/io.netty.netty-codec-4.1.50.Final.jar:lib/io.strimzi.config-model-0.20.0.jar:lib/io.netty.netty-codec-socks-4.1.50.Final.jar:lib/io.fabric8.kubernetes-model-batch-4.12.0.jar:lib/io.fabric8.kubernetes-model-extensions-4.12.0.jar:lib/com.mchange.c3p0-0.9.5.4.jar:lib/com.fasterxml.jackson.core.jackson-databind-2.10.4.jar:lib/jakarta.xml.bind.jakarta.xml.bind-api-2.3.2.jar:lib/io.fabric8.kubernetes-model-scheduling-4.12.0.jar:lib/io.fabric8.zjsonpatch-0.3.0.jar:lib/com.fasterxml.jackson.core.jackson-annotations-2.10.4.jar:lib/com.mchange.mchange-commons-java-0.2.15.jar:lib/io.fabric8.openshift-model-console-4.12.0.jar:lib/com.squareup.okhttp3.okhttp-3.12.6.jar:lib/io.netty.netty-transport-native-epoll-4.1.50.Final-linux-x86_64.jar:lib/com.fasterxml.jackson.dataformat.jackson-dataformat-yaml-2.10.4.jar:lib/io.netty.netty-handler-4.1.50.Final.jar:lib/io.fabric8.kubernetes-model-autoscaling-4.12.0.jar:lib/io.strimzi.crd-annotations-0.20.0.jar:lib/org.slf4j.slf4j-api-1.7.25.jar:lib/org.latencyutils.LatencyUtils-2.0.3.jar:lib/io.fabric8.kubernetes-model-admissionregistration-4.12.0.jar:lib/io.fabric8.kubernetes-model-policy-4.12.0.jar:lib/io.fabric8.kubernetes-model-rbac-4.12.0.jar:lib/org.apache.zookeeper.zookeeper-jute-3.5.8.jar:lib/io.fabric8.openshift-client-4.12.0.jar:lib/org.bouncycastle.bcpkix-jdk15on-1.65.jar:lib/com.fasterxml.jackson.datatype.jackson-datatype-jsr310-2.11.2.jar:lib/io.netty.netty-buffer-4.1.50.Final.jar:lib/org.apache.logging.log4j.log4j-core-2.13.3.jar:lib/io.netty.netty-codec-http-4.1.50.Final.jar:lib/io.micrometer.micrometer-registry-prometheus-1.3.1.jar:lib/org.quartz-scheduler.quartz-2.3.2.jar io.strimzi.operator.cluster.Main
2020-11-16 19:20:26 INFO  Main:60 - ClusterOperator 0.20.0 is starting
2020-11-16 19:20:28 INFO  Main:80 - Environment facts gathered: ClusterOperatorConfig(KubernetesVersion=1.15,OpenShiftRoutes=false,OpenShiftBuilds=false,OpenShiftImageStreams=false,OpenShiftDeploymentConfigs=false)
2020-11-16 19:20:28 INFO  Util:272 - Using config:
    PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
    STRIMZI_DEFAULT_KAFKA_BRIDGE_IMAGE: strimzi/kafka-bridge:0.19.0
    STRIMZI_CLUSTER_OPERATOR_SVC_PORT: tcp://172.20.124.225:80
    TINI_SHA256: 12d20136605531b09a2c2dac02ccee85e1b874eb322ef6baf7561cd93f93c855
    STRIMZI_KAFKA_MIRROR_MAKER_IMAGES: 2.5.0=strimzi/kafka:0.20.0-kafka-2.5.0
2.5.1=strimzi/kafka:0.20.0-kafka-2.5.1
2.6.0=strimzi/kafka:0.20.0-kafka-2.6.0

    KUBERNETES_PORT: tcp://172.20.0.1:443
    STRIMZI_DEFAULT_TOPIC_OPERATOR_IMAGE: strimzi/operator:0.20.0
    JAVA_OPTS:  -Dlog4j2.configurationFile=file:/opt/strimzi/custom-config/log4j2.properties -Dvertx.cacheDirBase=/tmp -Djava.security.egd=file:/dev/./urandom  --illegal-access=deny
    STRIMZI_FULL_RECONCILIATION_INTERVAL_MS: 120000
    STRIMZI_CLUSTER_OPERATOR_SVC_SERVICE_PORT: 80
    KUBERNETES_SERVICE_HOST: 172.20.0.1
    STRIMZI_CLUSTER_OPERATOR_SVC_PORT_80_TCP_PORT: 80
    STRIMZI_CLUSTER_OPERATOR_SVC_PORT_80_TCP: tcp://172.20.124.225:80
    JAVA_CLASSPATH: lib/io.strimzi.cluster-operator-0.20.0.jar:lib/com.fasterxml.jackson.core.jackson-core-2.10.4.jar:lib/io.prometheus.simpleclient_common-0.7.0.jar:lib/io.vertx.vertx-micrometer-metrics-3.9.1.jar:lib/io.fabric8.openshift-model-operator-4.12.0.jar:lib/org.apache.zookeeper.zookeeper-3.5.8.jar:lib/org.apache.logging.log4j.log4j-api-2.13.3.jar:lib/com.zaxxer.HikariCP-java7-2.4.13.jar:lib/org.keycloak.keycloak-core-11.0.2.jar:lib/io.fabric8.kubernetes-model-apps-4.12.0.jar:lib/org.bouncycastle.bcprov-jdk15on-1.60.jar:lib/com.squareup.okio.okio-1.15.0.jar:lib/io.fabric8.kubernetes-model-storageclass-4.12.0.jar:lib/org.keycloak.keycloak-common-11.0.2.jar:lib/io.netty.netty-codec-http2-4.1.50.Final.jar:lib/io.fabric8.kubernetes-model-coordination-4.12.0.jar:lib/io.fabric8.kubernetes-model-settings-4.12.0.jar:lib/io.strimzi.kafka-oauth-client-0.6.1.jar:lib/org.xerial.snappy.snappy-java-1.1.7.3.jar:lib/org.hdrhistogram.HdrHistogram-2.1.11.jar:lib/io.prometheus.simpleclient-0.7.0.jar:lib/com.fasterxml.jackson.module.jackson-module-jaxb-annotations-2.10.4.jar:lib/com.sun.activation.jakarta.activation-1.2.1.jar:lib/io.fabric8.kubernetes-model-events-4.12.0.jar:lib/com.squareup.okhttp3.logging-interceptor-3.12.12.jar:lib/javax.xml.bind.jaxb-api-2.3.0.jar:lib/org.apache.yetus.audience-annotations-0.5.0.jar:lib/io.fabric8.kubernetes-model-discovery-4.12.0.jar:lib/javax.annotation.javax.annotation-api-1.3.2.jar:lib/io.fabric8.kubernetes-model-apiextensions-4.12.0.jar:lib/io.fabric8.kubernetes-client-4.12.0.jar:lib/io.strimzi.api-0.20.0.jar:lib/io.netty.netty-handler-proxy-4.1.50.Final.jar:lib/io.netty.netty-codec-dns-4.1.50.Final.jar:lib/io.micrometer.micrometer-core-1.3.1.jar:lib/io.strimzi.certificate-manager-0.20.0.jar:lib/io.fabric8.kubernetes-model-certificates-4.12.0.jar:lib/io.fabric8.openshift-model-monitoring-4.12.0.jar:lib/io.strimzi.kafka-oauth-common-0.6.1.jar:lib/jakarta.activation.jakarta.activation-api-1.2.1.jar:lib/io.netty.netty-transport-native-epoll-4.1.50.Final.jar:lib/io.netty.netty-transport-4.1.50.Final.jar:lib/io.fabric8.kubernetes-model-metrics-4.12.0.jar:lib/com.github.mifmif.generex-1.0.2.jar:lib/io.fabric8.kubernetes-model-common-4.12.0.jar:lib/io.netty.netty-common-4.1.50.Final.jar:lib/io.strimzi.kafka-oauth-server-0.6.1.jar:lib/io.netty.netty-transport-native-unix-common-4.1.50.Final.jar:lib/io.vertx.vertx-core-3.9.1.jar:lib/com.github.luben.zstd-jni-1.4.4-7.jar:lib/io.netty.netty-resolver-dns-4.1.50.Final.jar:lib/org.yaml.snakeyaml-1.26.jar:lib/io.fabric8.kubernetes-model-core-4.12.0.jar:lib/io.fabric8.kubernetes-model-networking-4.12.0.jar:lib/io.strimzi.operator-common-0.20.0.jar:lib/io.fabric8.openshift-model-4.12.0.jar:lib/io.netty.netty-resolver-4.1.50.Final.jar:lib/org.lz4.lz4-java-1.7.1.jar:lib/org.apache.kafka.kafka-clients-2.6.0.jar:lib/io.fabric8.openshift-model-operatorhub-4.12.0.jar:lib/org.apache.logging.log4j.log4j-slf4j-impl-2.13.3.jar:lib/dk.brics.automaton.automaton-1.11-8.jar:lib/io.netty.netty-codec-4.1.50.Final.jar:lib/io.strimzi.config-model-0.20.0.jar:lib/io.netty.netty-codec-socks-4.1.50.Final.jar:lib/io.fabric8.kubernetes-model-batch-4.12.0.jar:lib/io.fabric8.kubernetes-model-extensions-4.12.0.jar:lib/com.mchange.c3p0-0.9.5.4.jar:lib/com.fasterxml.jackson.core.jackson-databind-2.10.4.jar:lib/jakarta.xml.bind.jakarta.xml.bind-api-2.3.2.jar:lib/io.fabric8.kubernetes-model-scheduling-4.12.0.jar:lib/io.fabric8.zjsonpatch-0.3.0.jar:lib/com.fasterxml.jackson.core.jackson-annotations-2.10.4.jar:lib/com.mchange.mchange-commons-java-0.2.15.jar:lib/io.fabric8.openshift-model-console-4.12.0.jar:lib/com.squareup.okhttp3.okhttp-3.12.6.jar:lib/io.netty.netty-transport-native-epoll-4.1.50.Final-linux-x86_64.jar:lib/com.fasterxml.jackson.dataformat.jackson-dataformat-yaml-2.10.4.jar:lib/io.netty.netty-handler-4.1.50.Final.jar:lib/io.fabric8.kubernetes-model-autoscaling-4.12.0.jar:lib/io.strimzi.crd-annotations-0.20.0.jar:lib/org.slf4j.slf4j-api-1.7.25.jar:lib/org.latencyutils.LatencyUtils-2.0.3.jar:lib/io.fabric8.kubernetes-model-admissionregistration-4.12.0.jar:lib/io.fabric8.kubernetes-model-policy-4.12.0.jar:lib/io.fabric8.kubernetes-model-rbac-4.12.0.jar:lib/org.apache.zookeeper.zookeeper-jute-3.5.8.jar:lib/io.fabric8.openshift-client-4.12.0.jar:lib/org.bouncycastle.bcpkix-jdk15on-1.65.jar:lib/com.fasterxml.jackson.datatype.jackson-datatype-jsr310-2.11.2.jar:lib/io.netty.netty-buffer-4.1.50.Final.jar:lib/org.apache.logging.log4j.log4j-core-2.13.3.jar:lib/io.netty.netty-codec-http-4.1.50.Final.jar:lib/io.micrometer.micrometer-registry-prometheus-1.3.1.jar:lib/org.quartz-scheduler.quartz-2.3.2.jar
    STRIMZI_KAFKA_CONNECT_S2I_IMAGES: 2.5.0=strimzi/kafka:0.20.0-kafka-2.5.0
2.5.1=strimzi/kafka:0.20.0-kafka-2.5.1
2.6.0=strimzi/kafka:0.20.0-kafka-2.6.0

    STRIMZI_CLUSTER_OPERATOR_SVC_PORT_80_TCP_PROTO: tcp
    TINI_VERSION: v0.18.0
    STRIMZI_HOME: /opt/strimzi
    STRIMZI_DEFAULT_TLS_SIDECAR_CRUISE_CONTROL_IMAGE: strimzi/kafka:0.20.0-kafka-2.6.0
    PWD: /opt/strimzi
    STRIMZI_DEFAULT_CRUISE_CONTROL_IMAGE: strimzi/kafka:0.20.0-kafka-2.6.0
    KUBERNETES_PORT_443_TCP: tcp://172.20.0.1:443
    STRIMZI_OPERATION_TIMEOUT_MS: 300000
    JAVA_MAIN: io.strimzi.operator.cluster.Main
    KUBERNETES_PORT_443_TCP_ADDR: 172.20.0.1
    STRIMZI_VERSION: 0.20.0
    STRIMZI_DEFAULT_JMXTRANS_IMAGE: strimzi/jmxtrans:0.20.0
    STRIMZI_NAMESPACE: kafka
    STRIMZI_KAFKA_MIRROR_MAKER_2_IMAGES: 2.5.0=strimzi/kafka:0.20.0-kafka-2.5.0
2.5.1=strimzi/kafka:0.20.0-kafka-2.5.1
2.6.0=strimzi/kafka:0.20.0-kafka-2.6.0

    STRIMZI_CLUSTER_OPERATOR_SVC_SERVICE_HOST: 172.20.124.225
    KUBERNETES_PORT_443_TCP_PROTO: tcp
    STRIMZI_DEFAULT_USER_OPERATOR_IMAGE: strimzi/operator:0.20.0
    KUBERNETES_SERVICE_PORT: 443
    STRIMZI_CLUSTER_OPERATOR_SVC_SERVICE_PORT_TCP_PROMETHEUS: 80
    STRIMZI_DEFAULT_KAFKA_EXPORTER_IMAGE: strimzi/kafka:0.20.0-kafka-2.6.0
    STRIMZI_DEFAULT_KAFKA_INIT_IMAGE: strimzi/operator:0.20.0
    HOSTNAME: strimzi-cluster-operator-8cd8fd67b-gqh2d
    STRIMZI_CLUSTER_OPERATOR_SVC_PORT_80_TCP_ADDR: 172.20.124.225
    STRIMZI_KAFKA_CONNECT_IMAGES: 2.5.0=strimzi/kafka:0.20.0-kafka-2.5.0
2.5.1=strimzi/kafka:0.20.0-kafka-2.5.1
2.6.0=strimzi/kafka:0.20.0-kafka-2.6.0

    STRIMZI_KAFKA_IMAGES: 2.5.0=strimzi/kafka:0.20.0-kafka-2.5.0
2.5.1=strimzi/kafka:0.20.0-kafka-2.5.1
2.6.0=strimzi/kafka:0.20.0-kafka-2.6.0

    KUBERNETES_PORT_443_TCP_PORT: 443
    STRIMZI_DEFAULT_TLS_SIDECAR_ENTITY_OPERATOR_IMAGE: strimzi/kafka:0.20.0-kafka-2.6.0
    KUBERNETES_SERVICE_PORT_HTTPS: 443
    SHLVL: 0
    HOME: /home/strimzi
    MALLOC_ARENA_MAX: 2

2020-11-16 19:20:28 INFO  Main:121 - The KafkaConnectS2I custom resource definition can only be used in environment which supports OpenShift build, image and apps APIs. These APIs do not seem to be supported in this environment.
2020-11-16 19:20:28 INFO  ClusterOperator:81 - Creating ClusterOperator for namespace kafka
2020-11-16 19:20:28 INFO  ClusterOperator:98 - Starting ClusterOperator for namespace kafka
2020-11-16 19:20:28 WARN  WatchConnectionManager:198 - Exec Failure: HTTP 404, Status: 404 - 404 page not found

(...)

java.net.ProtocolException: Expected HTTP 101 response but was '404 Not Found'
    at okhttp3.internal.ws.RealWebSocket.checkResponse(RealWebSocket.java:229) [com.squareup.okhttp3.okhttp-3.12.6.jar:?]
    at okhttp3.internal.ws.RealWebSocket$2.onResponse(RealWebSocket.java:196) [com.squareup.okhttp3.okhttp-3.12.6.jar:?]
    at okhttp3.RealCall$AsyncCall.execute(RealCall.java:203) [com.squareup.okhttp3.okhttp-3.12.6.jar:?]
    at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32) [com.squareup.okhttp3.okhttp-3.12.6.jar:?]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
    at java.lang.Thread.run(Thread.java:834) [?:?]
2020-11-16 19:20:30 ERROR Main:156 - Cluster Operator verticle in namespace kafka failed to start
io.fabric8.kubernetes.client.KubernetesClientException: 404 page not found

    at io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager$1.onFailure(WatchConnectionManager.java:202) ~[io.fabric8.kubernetes-client-4.12.0.jar:?]
    at okhttp3.internal.ws.RealWebSocket.failWebSocket(RealWebSocket.java:571) ~[com.squareup.okhttp3.okhttp-3.12.6.jar:?]
    at okhttp3.internal.ws.RealWebSocket$2.onResponse(RealWebSocket.java:198) ~[com.squareup.okhttp3.okhttp-3.12.6.jar:?]
    at okhttp3.RealCall$AsyncCall.execute(RealCall.java:203) ~[com.squareup.okhttp3.okhttp-3.12.6.jar:?]
    at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32) ~[com.squareup.okhttp3.okhttp-3.12.6.jar:?]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
    at java.lang.Thread.run(Thread.java:834) [?:?]
    Suppressed: java.lang.Throwable: waiting here
        at io.fabric8.kubernetes.client.utils.Utils.waitUntilReady(Utils.java:154) ~[io.fabric8.kubernetes-client-4.12.0.jar:?]
        at io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager.waitUntilReady(WatchConnectionManager.java:341) ~[io.fabric8.kubernetes-client-4.12.0.jar:?]
        at io.fabric8.kubernetes.client.dsl.base.BaseOperation.watch(BaseOperation.java:818) ~[io.fabric8.kubernetes-client-4.12.0.jar:?]
        at io.fabric8.kubernetes.client.dsl.base.BaseOperation.watch(BaseOperation.java:791) ~[io.fabric8.kubernetes-client-4.12.0.jar:?]
        at io.fabric8.kubernetes.client.dsl.base.BaseOperation.watch(BaseOperation.java:84) ~[io.fabric8.kubernetes-client-4.12.0.jar:?]
        at io.strimzi.operator.common.operator.resource.AbstractWatchableResourceOperator.watch(AbstractWatchableResourceOperator.java:63) ~[io.strimzi.operator-common-0.20.0.jar:0.20.0]
        at io.strimzi.operator.common.AbstractOperator.lambda$createWatch$16(AbstractOperator.java:429) ~[io.strimzi.operator-common-0.20.0.jar:0.20.0]
        at io.strimzi.operator.common.Util.lambda$async$0(Util.java:53) ~[io.strimzi.operator-common-0.20.0.jar:0.20.0]
        at io.vertx.core.impl.ContextImpl.lambda$executeBlocking$2(ContextImpl.java:313) ~[io.vertx.vertx-core-3.9.1.jar:3.9.1]
        at io.vertx.core.impl.TaskQueue.run(TaskQueue.java:76) ~[io.vertx.vertx-core-3.9.1.jar:3.9.1]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty.netty-common-4.1.50.Final.jar:4.1.50.Final]
        at java.lang.Thread.run(Thread.java:834) [?:?]

If i tried to deploy the previous version (0.19.0) of the helm chart, all works fine and the operator is deployed with success. I had dig a little on the differences between what is deployed between 0.19.0 and 0.20.0 and i notice that it appears that with v0.20.0 the CRD are not deployed.

Looking to the helm code i see, that is a property createGlobalResources that enable or not the deploy, but by default is value is already true.

https://github.com/strimzi/strimzi-kafka-operator/blob/master/helm-charts/helm2/strimzi-kafka-operator/templates/040-Crd-kafka.yaml

I'am deploying the operator with terraform helm provider. (v0.12.2)

## https://www.terraform.io/docs/providers/helm/repository.html
data "helm_repository" "strimzi" {
  name = "strimzi"
  url  = "https://strimzi.io/charts/"
}
## https://www.terraform.io/docs/providers/helm/release.html
resource "helm_release" "kafka-operator" {
  name       = "kafka-operator"
  chart      = "strimzi/strimzi-kafka-operator"
  repository = data.helm_repository.strimzi.metadata[0].name
  namespace  = var.controller_namespace
  version    = "0.20.0" 
}

So i get no idea, why they are not being deployed, causing the above issue.

Did i miss any value? or i need to do anything else on version 0.20.0? Any help would be appreciated. Thanks

scholzj commented 3 years ago

Did you used the Helm2 Chart from https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.20.0? Or just the Helm3 Chart from the index on the website? The Helm3 Chart has the CRDs in a subfolder or something like that so maybe Helm2 did not installed them?

(the error from the Cluster Operator basically says that the CRDs are not installed)

psamagal commented 3 years ago

Yes, i used the v0.2.0 helm release version. You can check my only code on the terraform snippet above.

What else should i do to install the CRD's? is there a separate deployment? Or do i need to go with Helm3?

scholzj commented 3 years ago

My understanding is that with Helm you should use the strimzi-kafka-operator-helm-2-chart-0.20.0.tgz chart. But I do not know much about Helm TBH within or without Terraform. But the idea definitely is that it installs the CRDs as well.

psamagal commented 3 years ago

That was what i was expecting, the CRD to be installed, and it worked well with v0.19.0 of the chart. I will try to enable debug on terraform helm provider to see if something pops up, during the apply. If not, I will try a manual install of the operator to see, if the issue persists.

scholzj commented 3 years ago

So back to my original question which I'm still not sure you answered. Did you used the Helm2 Chart from the release page or not? Because the index now links tot eh Helm 3 Chart as Helm2 is end-of-life.

psamagal commented 3 years ago

No, i did not download it from release page. i did only the terraform above, any download was made by terraform itself, will try to run in debug to find out, what terraform does in the background. This is the helm provider configuration

provider "kubernetes" {
  host                   = var.gke_cluster_api
  token                  = var.gke_cluster_token
  cluster_ca_certificate = var.gke_cluster_ca_certificate
  load_config_file       = false
}

provider "helm" {
  version        = "0.10.2"
  install_tiller = false
  kubernetes {
    host                   = var.gke_cluster_api
    token                  = var.gke_cluster_token
    cluster_ca_certificate = var.gke_cluster_ca_certificate
    config_context         = "none"
  }
}

Yes, is possible a good idea to start playing with helm 3. :D

psamagal commented 3 years ago

A quick test with helm 3 + minikube and most recent terraform provider for helm. it did work well. With Helm 2 + minikube + old terraform provider (max version, before helm3 was enforced) it was failing.

I will just update to Helm 3, luckily new version does not need tiller anymore, so i can upgrade projects 1 by 1. Will close this, don't believe it's worth it to chase the issue around, on old versions.

Thanks @scholzj for the help.