Are there any other ways to transfer large CRD files?

kubernetes-client / gen

Common generator scripts for all client libraries

Apache License 2.0

148 stars 146 forks source link

Are there any other ways to transfer large CRD files? #266

Open xiaoyu1095 opened 3 months ago

xiaoyu1095 commented 3 months ago

FILE="rayjobs.yaml" GROUP_NAME="io.ray" PACKAGE="io.ray"

docker run --rm -v pwd/src/main/resources/crd/${FILE}:pwd/src/main/resources/crd/${FILE} -v /var/run/docker.sock:/var/run/docker.sock -v /tmp/output:/tmp/output -v "$(pwd)":"$(pwd)" -ti --network host ghcr.io/kubernetes-client/java/crd-model-gen:v1.0.6 /generate.sh -u pwd/src/main/resources/crd/${FILE} -n ${GROUP_NAME} -p ${PACKAGE} -o "$(pwd)"

The CustomResourceDefinition "rayjobs.ray.io" is invalid: metadata.annotations: Too long: must have at most 262144 bytes ps: crd file : https://github.com/ray-project/kuberay/blob/master/ray-operator/config/crd/bases/ray.io_rayjobs.yaml

brendandburns commented 3 months ago

I think something is going wrong with the parsing here. I looked at your CRD, the annotations field isn't that long, I'm wondering if the YAML is messed up somehow.

xiaoyu1095 commented 3 months ago

How is YAML transmitted? The file size is 1.2 MB, but Kubernetes label limits are usually 0.25 MB.

brendandburns commented 2 months ago

Ahh, I think that we must be using kubectl apply somewhere. That tries to shove the old object representation into an annotation (which then triggers this)

If we switch to kubectl create in that script it probably will work correctly. I don't think there's any reason to use apply

Any chance you can try making that change and seeing if it works?