Fabric8 Kubernetes Client mismatch

[Donnerbart]

Hey folks,

I'm seeing this warning with at least the most recent versions of the Quarkus Operator SDK dependency (6.6.8 and 6.7.0).

With 6.7.0 the warning is:

WARN  [io.qua.ope.dep.VersionAlignmentCheckingStep] (build-46) Mismatched JOSDK Fabric8 Kubernetes
Client version found: "6.11.0", expected: "6.12.1" by at least a minor version and things might not
work as expected.

When calling mvn dependency:tree on the java-operator-sdk I get version 6.12.1 for all Fabric8 dependencies. On quarkus-operator-sdk it's 6.11.0.

I'm actually a bit lost where the 6.11.0 in this project is coming from. I tried to follow all parent BOMs, but I cannot find this version anywhere. Nevertheless I think the root cause is somewhere in this project.

[metacosm]

Hi @Donnerbart, which version of Quarkus are you using?

[Donnerbart]

From our build.gradle.kts:

    // Quarkus Operator SDK
    implementation(enforcedPlatform("io.quarkiverse.operatorsdk:quarkus-operator-sdk-bom:${property("quarkus-operator-sdk.version")}"))
    implementation("io.quarkiverse.operatorsdk:quarkus-operator-sdk")
    implementation("io.quarkiverse.operatorsdk:quarkus-operator-sdk-bundle-generator")

    // Quarkus Logging
    implementation("io.quarkiverse.logging.logback:quarkus-logging-logback:${property("quarkus-logging-logback.version")}")

    // Quarkus
    implementation("io.quarkus:quarkus-arc")
    implementation("io.quarkus:quarkus-cache")
    implementation("io.quarkus:quarkus-container-image-docker")
    implementation("io.quarkus:quarkus-micrometer-registry-prometheus")
    implementation("io.quarkus:quarkus-resteasy-reactive-jackson")

    // Quarkus Deployment
    // set property using ~/.gradle/gradle.properties
    if (project.findProperty("operator-deployment") == "kind") {
        logger.info("Adding conditional dependency 'quarkus-kind'")
        implementation("io.quarkus:quarkus-kind")
    }
    if (project.findProperty("operator-deployment") == "minikube") {
        logger.info("Adding conditional dependency 'quarkus-minikube'")
        implementation("io.quarkus:quarkus-minikube")
    }

    // K8s Webhooks
    implementation("io.javaoperatorsdk:kubernetes-webhooks-framework-core:${property("kubernetes-webhooks-framework.version")}")

    // misc
    implementation("org.bouncycastle:bcpkix-jdk15on:${property("bouncycastle.version")}")
    implementation("org.bouncycastle:bcprov-jdk15on:${property("bouncycastle.version")}")
    compileOnly("org.jetbrains:annotations:${property("jetbrains-annotations.version")}")
    compileOnly("io.fabric8:crd-generator-api")

This is from our gradle.properties:

# keep in sync with the Operator SDK BOM version (https://quarkiverse.github.io/quarkiverse-docs/quarkus-operator-sdk/dev/index.html#_installation)
# see https://central.sonatype.com/artifact/io.quarkiverse.operatorsdk/quarkus-operator-sdk-parent
quarkus.version=3.10.1

# see https://central.sonatype.com/artifact/io.quarkiverse.operatorsdk/quarkus-operator-sdk-bom
# see https://github.com/quarkiverse/quarkus-operator-sdk/releases
# see https://github.com/operator-framework/java-operator-sdk/releases
quarkus-operator-sdk.version=6.7.0

We even disabled Renovate for these, so we can assure to keep the Quarkus version in sync with the BOM version.

[Donnerbart]

But as I wrote, I can already see the mismatch in the versions when I check out the 6.7.0 tag on the quarkus-operator-sdk project and run mvn dependency:tree (reports 6.11.0). And checkout the tag v4.9.0 on the java-operator-sdk project (reports 6.12.1).

[metacosm]

OK, this warning is normal. The client version that gets picked up by quarkus-operator-sdk is the version coming from Quarkus itself. JOSDK 4.9.0 indeed uses the latest version (6.12.1 as of this writing) but Quarkus 3.10.1 is still using 6.11. This shouldn't be an issue but do let us know if you encounter any.

[Donnerbart]

Then I don't understand why the VersionAlignmentCheckingStep is part of quarkus-operator-sdk (even with an option to fail the build on a mismatch, if I read the code correctly), when this project isn't also ensuring that the same version is packaged.

Is it no option to align the packaged version in quarkus-operator-sdk and ship whatever java-operator-sdk is using?

[metacosm]

The alignment is done more on Quarkus than JOSDK, actually, since this is the most tricky aspect. The version check is done here because that's the only spot where it can be done. JOSDK is not too sensitive about which version of the client is used, which isn't the case for Quarkus when you intend to compile natively. Hopefully, this makes more sense now. This check is meant to be a hint if people encounter issues, which had happened several times, due to mismatched versions. It is possible to override the fabric8 client version used in QOSDK if you wish but we don't recommend it. Are you encountering a specific issue?

[Donnerbart]

The explanation with the native image makes a lot sense, thank you.

We don't have a concrete production issue I can relate to this. I'm just very strict with warnings in the build process and tests. I try to get rid of them as much as possible, since they normally indicate that something is wrong (and might break in the future).

So if I understood this correctly, it's more important that the Fabric8 version from the Quarkus BOM is used than the one from the Java Operator SDK. I'm pretty sure I'm doing this, I'm even enforcing this via implementation(enforcedPlatform("io.quarkiverse.operatorsdk:quarkus-operator-sdk-bom:${property("quarkus-operator-sdk.version")}")).

I think in this case there should be no warning logged, even if there is a mismatch between the Quarkus BOM and the Java Operator SDK. The warning should only be logged when the Fabric8 version differs from the Quarkus BOM.

[metacosm]

The explanation with the native image makes a lot sense, thank you.

Glad it helped.

We don't have a concrete production issue I can relate to this. I'm just very strict with warnings in the build process and tests. I try to get rid of them as much as possible, since they normally indicate that something is wrong (and might break in the future).

Gotcha. I tend to be the same way, when I can :)

So if I understood this correctly, it's more important that the Fabric8 version from the Quarkus BOM is used than the one from the Java Operator SDK. I'm pretty sure I'm doing this, I'm even enforcing this via implementation(enforcedPlatform("io.quarkiverse.operatorsdk:quarkus-operator-sdk-bom:${property("quarkus-operator-sdk.version")}")).

Yes, it is usually more important to use the same client version as the one defined by Quarkus (hence coming from the Quarkus BOM)…

I think in this case there should be no warning logged, even if there is a mismatch between the Quarkus BOM and the Java Operator SDK. The warning should only be logged when the Fabric8 version differs from the Quarkus BOM.

… however, there had been instances where the client version used by JOSDK was more important (in case, we needed a specific fix in the client as soon as possible and the Quarkus version hadn't yet caught up, for example). That said, this hasn't happened in a long time and occurred more when the project was less mature when we introduced specific features in the client (like the CRD generator) specifically for JOSDK. This is the reason why we prefer to warn if we detect a mismatch at any level. They are usually inconsequential but should help figure issues out when they occur and a mismatch exists.