AdmissionReview deserialization issue

[bachmanity1]

I'm using AdmissionReview to implement webhook, it used to work well in version 6.2.0 but after upgrading to version 6.5.0 I started facing issues.

{
  "apiVersion": "admission.k8s.io/v1",
  "kind": "AdmissionReview",
  "request": {
    "object": {
       // my custom resource
    },
    "name": "test",
    "namespace": "test",
    "uid": "uid",
    "operation": "CREATE"
  }
}

In version 6.2.0, when I sent a json shown above to the webhook server request.object field was deserialized as a CustomResource class, however in version 6.5.0 this same field is deserialized as a GenericKubernetesResource. I haven't changed anything else just downgrading kubernetes-client version to 6.2.0 resolves the problem. How can I achieve same behavior using version 6.5.0?

[shawkins]

Earlier releases had a side effect of registering custom classes with the deserializer. This was removed to prevent situations like #5012 More than likely the resolution for you will be to register your custom class with the deserializer. See #3923 the supported way of doing this is including a /META-INF/services/io.fabric8.kubernetes.api.model.KubernetesResource with your custom resources. We are working towards offering more supported ways of doing this in subsequent releases.

[bachmanity1]

I'm facing exactly the same issue as described here. However, adding resources/META-INF/services/io.fabric8.kubernetes.api.model.KubernetesResource file with content as shown below didn't resolve my issue.

com.operator.model.MyCustomResource

I've also tried to register my custom resource using KubernetesDeserializer during operator initialization like shown below:

KubernetesDeserializer.registerCustomKind(HasMetadata.getKind(MyCustomResource.class),HasMetadata.getApiVersion(MyCustomResource.class), MyCustomResource.class);

but it also didn't help.

[shawkins]

@bachmanity1 from here the issue will lie with the rest of the application - the KubernetesDeserializer that is doing the deserialization must be being created in an isolated classloader - not the same one as you are registering from, nor one that is seeing the META-INF/services. I believe @csviri saw a situation like that before in the operator sdk.

[csviri]

Hi @bachmanity1 , pls take a look on this project: https://github.com/java-operator-sdk/kubernetes-webooks-framework It might also help you to implement the webhook.

There are samples (not for webhook, but conversion hook, which essentially work the same way in regards the deserialization).

In the most recent release the approach adding the file into: resources/META-INF/services/io.fabric8.kubernetes.api.model.KubernetesResource is used.

See: https://github.com/java-operator-sdk/kubernetes-webooks-framework/blob/main/samples/commons/src/main/resources/META-INF/services/io.fabric8.kubernetes.api.model.KubernetesResource

[bachmanity1]

The problem was that I didn't have a @Kind annotation on my custom class, adding this annotation and registering custom class with KubernetesDeserializer resolved the issue. However, registering custom class using resources/META-INF/services/io.fabric8.kubernetes.api.model.KubernetesResource file still doesn't work.

[shawkins]

However, registering custom class using resources/META-INF/services/io.fabric8.kubernetes.api.model.KubernetesResource file still doesn't work

There are two possibilities. Either you are dealing with a KubernetesDeserializer in an isolated classloader. Can you set a breakpoint in the KubernetesDeserializer - https://github.com/fabric8io/kubernetes-client/blob/009974fe5fe634cbd6d1210780b99ff82e251f2d/kubernetes-model-generator/kubernetes-model-core/src/main/java/io/fabric8/kubernetes/internal/KubernetesDeserializer.java#L163

Do you see it being called multiple times in your application? This is not expected with quarkus as there's a flat classloader and just a single static instance of Mapping expected.

Or if it's only called a single time, then check with getKeyFromClass method if you class is passed to that - is it failing to add a mapping because apiGroup or apiVerion are missing?

[stale[bot]]

This issue has been automatically marked as stale because it has not had any activity since 90 days. It will be closed if no further activity occurs within 7 days. Thank you for your contributions!

[bachmanity1]

Hi @shawkins, I've started to face this issue again since version 6.7.0, I think this is caused by the changes made in the https://github.com/fabric8io/kubernetes-client/pull/4662.

Since version 6.5.0 I've registered my custom resources using the method shown below (this method is called in the main method when operator is initialized).

    @SafeVarargs
    public static void registerCustomKinds(Class<? extends KubernetesResource>... customKinds) {
        for (var customKind : customKinds) {
            KubernetesDeserializer.registerCustomKind(HasMetadata.getApiVersion(customKind),
                HasMetadata.getKind(customKind), customKind);
        }
    }

and everything worked fine. Then after the API changes made in #4662 I've modified my method as shown below:

    @SafeVarargs
    public static void registerCustomKinds(Class<? extends KubernetesResource>... customKinds) {
        final var deserializer = new KubernetesDeserializer();
                 // final var deserializer = new KubernetesDeserializer(false); also doesn't work
        for (var customKind : customKinds) {
            deserializer.registerCustomKind(HasMetadata.getApiVersion(customKind),
                HasMetadata.getKind(customKind), customKind);
                         // deserializer.registerKubernetesResource(customKind); also doesn't work
        }
    }

and now it doesn't work and I'm seeing the same error as before, i.e. CustomResource is deserialized as GenericKubernetesResource. This is the error message: java.lang.ClassCastException: class io.fabric8.kubernetes.api.model.GenericKubernetesResource cannot be cast to class com.myoperator.model.MyCustomResource (io.fabric8.kubernetes.api.model.GenericKubernetesResource and com.myoperator.model.MyCustomResource are in unnamed module of loader 'app')

I've also tried replacing KubernetesDeserializer with KubernetesSerialization but it also doesn't work.

    @SafeVarargs
    public static void registerCustomKinds(Class<? extends KubernetesResource>... customKinds) {
                 final var serializer = new KubernetesSerialization(); 
        // final var serializer = new KubernetesSerialization(new ObjectMapper(), false); also doesn't work
        for (var customKind : customKinds) {
            serializer.registerKubernetesResource(customKind);
        }
    }

I've also tried registering a custom resource using resources/META-INF/services/io.fabric8.kubernetes.api.model.KubernetesResource file but it also doesn't work (couldn't make it work even in version 6.5.0)

[shawkins]

KubernetesSerialization is passed to the KubernetesClient via the KubernetesClientBuilder if you want to customize prior to using the client, and you may access it on the client via client.getKubernetesSerialization.

[bachmanity1]

Yes, after reading #4662 more carefully I've figured out that KubernetesSerialization instance must be passed to the KubernetesClient but this doesn't work for me because in my case deserialization is not handled by the KubernetesClient but by the Spring Boot framework, i.e. AdmissionReview instance is received as a request body.

I could resolve my problem by serializing GenericKubernetesResource back to the json string and then deserializing it to the MyCustomResource.

Serialization.unmarshal(Serialization.asJson(gkr), MyCustomResource.class)

[shawkins]

KubernetesClient but this doesn't work for me because in my case deserialization is not handled by the KubernetesClient but by the Spring Boot framework, i.e. AdmissionReview instance is received as a request body

One approach would be to share the mapper used by KubernetesSerialization with spring boot.

[bachmanity1]

One approach would be to share the mapper used by KubernetesSerialization with spring boot.

How can I do it? mapper field is private and getMapper method is package private.

[shawkins]

You can pass a mapper in the KubernetesSerialization constructor.

[bachmanity1]

Sharing a mapper between KubernetesSerialization and spring boot does indeed prove to be effective. Thanks!