Arnavion / k8s-openapi

Rust definitions of the resource types in the Kubernetes client API
Apache License 2.0
386 stars 41 forks source link

How to explicitly not set a field? #158

Closed banool closed 3 weeks ago

banool commented 3 weeks ago

I am creating a Deployment and a HPA. I'm trying to set the replicas to None in the deployment to declare that I don't want to actually set a value for it when I create / update the resource:

    let deployment = Deployment {
        metadata: ObjectMeta {
            name: Some(name.to_string()),
            labels: Some(labels.clone()),
            ..Default::default()
        },
        spec: Some(DeploymentSpec {
            selector: LabelSelector {
                match_labels: Some(labels.clone()),
                ..Default::default()
            },
            // By default replica count is 1. We explicitly set it to None so that the
            // horizontal pod autoscaler is in full control of the replica count.
            replicas: None,
            ...
        }
    }
};

I then have a HPA like this:

fn create_hasura_hpa(_data: &instance_and_spec::Data) -> Result<HorizontalPodAutoscaler> {
    let name = get_hasura_name();
    let labels = get_labels();

    let hpa = HorizontalPodAutoscaler {
        metadata: ObjectMeta {
            name: Some(name.to_string()),
            labels: Some(labels.clone()),
            ..Default::default()
        },
        spec: Some(HorizontalPodAutoscalerSpec {
            scale_target_ref: CrossVersionObjectReference {
                api_version: Some("apps/v1".to_string()),
                kind: "Deployment".to_string(),
                name: name.to_string(),
            },
            min_replicas: Some(2),
            max_replicas: 10,
            metrics: Some(vec![MetricSpec {
                type_: "Resource".to_string(),
                resource: Some(ResourceMetricSource {
                    name: "cpu".to_string(),
                    target: MetricTarget {
                        type_: "Utilization".to_string(),
                        average_utilization: Some(80),
                        ..Default::default()
                    },
                }),
                ..Default::default()
            }]),
            ..Default::default()
        }),
        ..Default::default()
    };

    Ok(hpa)
}

When these resources are updated with the above specs, the target replicas for the Deployment becomes 1 again. The docs imply that I can set None to indicate that this is explicitly not set:

/// Number of desired pods. This is a pointer to distinguish between explicit zero and not specified. Defaults to 1.
pub replicas: Option<i32>,

But maybe I'm misinterpreting it. If so, is there a way to skip the replicas field when I update the resource?

banool commented 3 weeks ago

I see in the generated code that if replicas is None then it won't even serialize the field (I think), so I suppose everything is working as expected and the issue is elsewhere:

        if let Some(value) = &self.replicas {
            crate::serde::ser::SerializeStruct::serialize_field(&mut state, "replicas", value)?;
        }

Just weird that every time I update the resource the spec changes to have replicas: 1 again, with the HPA then setting it back to 2, over and over.

I'm getting sort of conflicting information. This source says that you should omit the field, but it also says that the default field is 1: https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/#migrating-deployments-and-statefulsets-to-horizontal-autoscaling. Anyway, the thing relevant to this crate is whether it is truly leaving replicas unspecified when making the request to the API. The code would imply as such, but when my code runs, the value is explicitly set regardless. So I suppose I need some way to explicitly not set the field rather than rely on the default behavior?

Arnavion commented 3 weeks ago

Yes, in k8s-openapi, a None field won't be serialized in the request at all. And yes, Kubernetes documents DeploymentSpec as setting replicas to 1 if it's not specified, so the API server will do that if you create a Deployment with the field set to None.

As for updates, when you update the Deployment later and again don't specify the replicas field, the API server might naively do the same thing and reset replicas to 1 even though an HPA is in effect, until the HPA runs again and resets it back to the scaled value. You'll have to check Kubernetes source to be sure, but I can imagine the Deployment controller is independent of the HPA controller so this behavior is expected.

banool commented 3 weeks ago

Okay got it, thanks for the clarification! I'll investigate the k8s behavior on my side.

Arnavion commented 3 weeks ago

Also note that the update strategy might matter, ie whether you're doing strategic merge or JSON merge.

banool commented 3 weeks ago

The issue is I was just using the wholesale replace function, I should be using patch. Thanks!