search

operator-framework / java-operator-sdk

Java SDK for building Kubernetes Operators
https://javaoperatorsdk.io/
Apache License 2.0
795 stars 214 forks source link

Dev mode operator fails with Kind cluster #1157

Closed bentito closed 2 years ago

bentito commented 2 years ago

Bug Report

What did you do?

Maven quarkus dev mode fails when trying to test operator on Kind cluster.

What did you expect to see?

mvn quarkus:dev should work.

What did you see instead? Under which circumstances?

On a Kind cluster, mvn quarkus:dev fails as follows:

trino-operator main $ mvn quarkus:dev
[INFO] Scanning for projects...
[INFO]
[INFO] --------------------------< tech.tofel:trino >--------------------------
[INFO] Building trino 0.0.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- quarkus-maven-plugin:2.7.1.Final:dev (default-cli) @ trino ---
[INFO] Invoking org.apache.maven.plugins:maven-resources-plugin:2.6:resources) @ trino
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Invoking org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile) @ trino
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 4 source files to /Users/btofel/workspace/trino-operator/target/classes
[INFO] Invoking org.apache.maven.plugins:maven-resources-plugin:2.6:testResources) @ trino
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /Users/btofel/workspace/trino-operator/src/test/resources
[INFO] Invoking org.apache.maven.plugins:maven-compiler-plugin:3.8.1:testCompile) @ trino
[INFO] No sources to compile
Listening for transport dt_socket at address: 5005
2022-04-13 14:23:52,684 INFO  [io.qua.ope.dep.OperatorSDKProcessor] (build-4) Registered 'tech.tofel.TrinoOperator' for reflection
2022-04-13 14:23:52,688 INFO  [io.qua.ope.dep.OperatorSDKProcessor] (build-4) Registered 'tech.tofel.TrinoOperatorSpec' for reflection
2022-04-13 14:23:52,688 INFO  [io.qua.ope.dep.OperatorSDKProcessor] (build-4) Registered 'tech.tofel.TrinoOperatorStatus' for reflection
2022-04-13 14:23:52,830 INFO  [io.qua.ope.dep.OperatorSDKProcessor] (build-4) Processed 'tech.tofel.TrinoOperatorController' reconciler named 'trinooperator' for 'trino-operators.trino-group.tofel.tech' resource (version 'trino-group.tofel.tech/v1alpha1')
2022-04-13 14:23:52,833 INFO  [io.fab.crd.gen.CRDGenerator] (build-4) Generating 'trino-operators.trino-group.tofel.tech' version 'v1alpha1' with tech.tofel.TrinoOperator (spec: tech.tofel.TrinoOperatorSpec / status tech.tofel.TrinoOperatorStatus)...
2022-04-13 14:23:52,992 INFO  [io.qua.ope.dep.OperatorSDKProcessor] (build-4) Generated trino-operators.trino-group.tofel.tech CRD:
2022-04-13 14:23:52,993 INFO  [io.qua.ope.dep.OperatorSDKProcessor] (build-4)   - v1 -> /Users/btofel/workspace/trino-operator/target/kubernetes/trino-operators.trino-group.tofel.tech-v1.yml
[
  [34;1mINFO [m] Checking for existing resources in: /Users/btofel/workspace/trino-operator/src/main/kubernetes.

2022-04-13 14:23:54,490 INFO  [io.qua.ope.csv.dep.CSVProcessor] (build-41) Generating CSV for trinooperator controller -> /Users/btofel/workspace/trino-operator/target/kubernetes/trinooperator.csv.yml
__  ____  __  _____   ___  __ ____  ______
 --/ __ \/ / / / _ | / _ \/ //_/ / / / __/
 -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/
2022-04-13 14:23:55,691 INFO  [io.qua.ope.run.OperatorProducer] (Quarkus Main Thread) Applied v1 CRD named 'trino-operators.trino-group.tofel.tech' from /Users/btofel/workspace/trino-operator/target/kubernetes/trino-operators.trino-group.tofel.tech-v1.yml

2022-04-13 14:23:55,695 INFO  [io.jav.ope.Operator] (Quarkus Main Thread) Registered reconciler: 'trinooperator' for resource: 'class tech.tofel.TrinoOperator' for namespace(s): [null]
2022-04-13 14:23:55,696 INFO  [io.qua.ope.run.AppEventListener] (Quarkus Main Thread) Quarkus Java Operator SDK extension 3.0.3 (commit: ccc257b on branch: ccc257b4706b455e82e5541d456c5a1592316ac4) built on Mon Feb 07 09:13:36 EST 2022
2022-04-13 14:23:55,696 INFO  [io.jav.ope.Operator] (Quarkus Main Thread) Operator SDK 2.1.1 (commit: 817f8ca) built on Mon Feb 07 05:16:44 EST 2022 starting...
2022-04-13 14:23:55,697 INFO  [io.jav.ope.Operator] (Quarkus Main Thread) Client version: 5.12.1
2022-04-13 14:23:55,724 WARN  [io.jav.ope.pro.eve.EventSourceManager] (Quarkus Main Thread) Error starting io.javaoperatorsdk.operator.processing.event.source.controller.ControllerResourceEventSource@70822e0e: java.lang.NullPointerException
    at java.base/java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
    at java.base/java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
    at io.javaoperatorsdk.operator.processing.event.source.controller.ControllerResourceEventSource.createAndRunInformerFor(ControllerResourceEventSource.java:107)
    at io.javaoperatorsdk.operator.processing.event.source.controller.ControllerResourceEventSource.lambda$start$0(ControllerResourceEventSource.java:89)
    at java.base/java.util.Collections$SingletonSet.forEach(Collections.java:4797)
    at io.javaoperatorsdk.operator.processing.event.source.controller.ControllerResourceEventSource.start(ControllerResourceEventSource.java:88)
    at io.javaoperatorsdk.operator.processing.event.EventSourceManager.start(EventSourceManager.java:72)
    at io.javaoperatorsdk.operator.processing.Controller.start(Controller.java:196)
    at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
    at java.base/java.util.HashMap$ValueSpliterator.forEachRemaining(HashMap.java:1675)
    at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
    at java.base/java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290)
    at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:746)
    at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
    at java.base/java.util.concurrent.ForkJoinTask.doInvoke(ForkJoinTask.java:408)
    at java.base/java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:736)
    at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:159)
    at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:173)
    at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
    at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
    at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:661)
    at io.javaoperatorsdk.operator.Operator$ControllerManager.start(Operator.java:164)
    at io.javaoperatorsdk.operator.Operator.start(Operator.java:82)
    at io.quarkiverse.operatorsdk.runtime.AppEventListener.onStartup(AppEventListener.java:31)
    at io.quarkiverse.operatorsdk.runtime.AppEventListener_Observer_onStartup_8d7b338c7415a5b0bcaaf3b82fd300311baae948.notify(Unknown Source)
    at io.quarkus.arc.impl.EventImpl$Notifier.notifyObservers(EventImpl.java:320)
    at io.quarkus.arc.impl.EventImpl$Notifier.notify(EventImpl.java:302)
    at io.quarkus.arc.impl.EventImpl.fire(EventImpl.java:73)
    at io.quarkus.arc.runtime.ArcRecorder.fireLifecycleEvent(ArcRecorder.java:128)
    at io.quarkus.arc.runtime.ArcRecorder.handleLifecycleEvents(ArcRecorder.java:97)
    at io.quarkus.deployment.steps.LifecycleEventsBuildStep$startupEvent1144526294.deploy_0(Unknown Source)
    at io.quarkus.deployment.steps.LifecycleEventsBuildStep$startupEvent1144526294.deploy(Unknown Source)
    at io.quarkus.runner.ApplicationImpl.doStart(Unknown Source)
    at io.quarkus.runtime.Application.start(Application.java:101)
    at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:103)
    at io.quarkus.runtime.Quarkus.run(Quarkus.java:67)
    at io.quarkus.runtime.Quarkus.run(Quarkus.java:41)
    at io.quarkus.runtime.Quarkus.run(Quarkus.java:120)
    at io.quarkus.runner.GeneratedMain.main(Unknown Source)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at io.quarkus.runner.bootstrap.StartupActionImpl$1.run(StartupActionImpl.java:103)
    at java.base/java.lang.Thread.run(Thread.java:834)

2022-04-13 14:23:55,801 INFO  [io.quarkus] (Quarkus Main Thread) trino 0.0.1-SNAPSHOT on JVM (powered by Quarkus 2.7.1.Final) started in 4.267s. Listening on: http://localhost:8080
2022-04-13 14:23:55,803 INFO  [io.quarkus] (Quarkus Main Thread) Profile dev activated. Live Coding activated.
2022-04-13 14:23:55,803 INFO  [io.quarkus] (Quarkus Main Thread) Installed features: [cdi, kubernetes, kubernetes-client, openshift-client, operator-sdk, smallrye-context-propagation, smallrye-health, vertx]

--
Tests paused
Press [r] to resume testing, [o] Toggle test output, [:] for the terminal, [h] for more options>

Same code works when used with dev mode with another type of cluster, for instance, CRC, minikube, etc.

Environment

Kubernetes cluster type:

Kind 0.12.0 (on MacOS with Homebrew, brew install kind)

$ Mention java-operator-sdk version from pom.xml file

3.0.3 <-- JOSDK pulled in as dep from this `$ java -version` ``` openjdk version "11.0.10" 2021-01-19 OpenJDK Runtime Environment GraalVM CE 21.0.0 (build 11.0.10+8-jvmci-21.0-b06) OpenJDK 64-Bit Server VM GraalVM CE 21.0.0 (build 11.0.10+8-jvmci-21.0-b06, mixed mode, sharing) ``` `$ kubectl version` ``` Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.5", GitCommit:"c285e781331a3785a7f436042c65c5641ce8a9e9", GitTreeState:"clean", BuildDate:"2022-03-16T15:51:05Z", GoVersion:"go1.17.8", Compiler:"gc", Platform:"darwin/amd64"} Server Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.4", GitCommit:"e6c093d87ea4cbb530a7b2ae91e54c0842d8308a", GitTreeState:"clean", BuildDate:"2022-03-06T21:32:53Z", GoVersion:"go1.17.7", Compiler:"gc", Platform:"linux/amd64"} ``` #### Possible Solution #### Additional context # reproducer notes: `git clone git@github.com:bentito/trino-operator.git` on MacOS: `$ brew install kind` as of Kind version 0.12.0 `$ mvn quarkus:dev` See error as supplied above.
metacosm commented 2 years ago

Still not the proper spot 😅 but let's keep it here. The issue, I think, is due to the fact that recent Fabric8 client version seem to be losing the current context. If you use a tool such as kubens or oc project to make sure you have an active/selected namespace/project before running your operator, the issue should go away. More recent versions of the SDK actually tells you about it when this occurs.

bentito commented 2 years ago

Ah indeed, oc project default failed to work easily for a Kind cluster, but tried: kubectl config set-context --current --namespace=default and it worked, so that's an easy workaround. Thanks!

metacosm commented 2 years ago

Yes, this didn't use to be such an issue before, though… Not sure why it's the case now.