search

kubernetes-client / java

Official Java client library for kubernetes
http://kubernetes.io/
Apache License 2.0
3.48k stars 1.86k forks source link

Java Model Generation Failed due to error message "The CustomResourceDefinition "xxx" is invalid: metadata.annotations: Too long: must have at most 262144 bytes" #2619

Closed hippoz closed 1 month ago

hippoz commented 1 year ago

Describe the bug Java Model Generation Failed due to the following error:

The CustomResourceDefinition "fleets.agones.dev" is invalid: metadata.annotations: Too long: must have at most 262144 bytes

Client Version e.g. 1.25.1

Kubernetes Version e.g. 1.25.1

Java Version e.g. Java 11

To Reproduce This bug is about crd-model-gen

Version: 1.06, 1.05 and 1.04

Steps to reproduce the behavior:

Expected behavior The models can be generated successfully for long CRD definitions

Workaround We can workaround this issue by remove all the descriptions from the CRD to make it shorter, but we lost all the java docs after the generation.

KubeConfig If applicable, add a KubeConfig file with secrets redacted.

Server (please complete the following information):

Additional context Add any other context about the problem here.

hippoz commented 1 year ago

This issue can be fixed by updating https://github.com/kubernetes-client/java/blob/34a7a2d19208bb140c0949054f14e67f00ad52d8/client-java-contrib/generate.sh#L52

from kubectl apply -f "$url" to kubectl create -f "$url"

k8s-triage-robot commented 1 year ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

scprek commented 6 months ago

Also hit this issue, not sure if the image has the latest generate script?

Also seems like the default image should move to ghcr.io/kubernetes-client/java/crd-model-gen to align with documentation https://github.com/kubernetes-client/java/blob/master/docs/generate-model-from-third-party-resources.md

❯ ./update.sh
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
Creating cluster "kind" ...
 â€ĸ Ensuring node image (kindest/node:v1.21.1) đŸ–ŧ  ...
 ✓ Ensuring node image (kindest/node:v1.21.1) đŸ–ŧ
 â€ĸ Preparing nodes đŸ“Ļ   ...
 ✓ Preparing nodes đŸ“Ļ
 â€ĸ Writing configuration 📜  ...
 ✓ Writing configuration 📜
 â€ĸ Starting control-plane 🕹ī¸  ...
 ✓ Starting control-plane 🕹ī¸
 â€ĸ Installing CNI 🔌  ...
 ✓ Installing CNI 🔌
 â€ĸ Installing StorageClass 💾  ...
 ✓ Installing StorageClass 💾
Set kubectl context to "kind-kind"
You can now use your cluster with:

kubectl cluster-info --context kind-kind

Thanks for using kind! 😊
customresourcedefinition.apiextensions.k8s.io/alertmanagers.monitoring.coreos.com created
customresourcedefinition.apiextensions.k8s.io/thanosrulers.monitoring.coreos.com created
customresourcedefinition.apiextensions.k8s.io/podmonitors.monitoring.coreos.com created
The CustomResourceDefinition "prometheuses.monitoring.coreos.com" is invalid: metadata.annotations: Too long: must have at most 262144 bytes
scprek commented 6 months ago

/remove-lifecycle stale

scprek commented 6 months ago

This is the generate script in the latest image v1.0.6 it does not have the latest changes to the generate.sh file including https://github.com/kubernetes-client/java/pull/2670.

❯ docker run -ti ghcr.io/kubernetes-client/java/crd-model-gen:v1.0.6 sh
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
/gen/openapi # cat /generate.sh
#!/bin/bash

# Copyright 2020 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# This script orchestrates the code generation procedure. It is executed inside a crdgen
# container in order to minimize the environment dependencies on the host, being Docker only.

CRD_URLS=${CRD_URLS:-}
OUTPUT_DIR=${OUTPUT_DIR:-}
KUBERNETES_CRD_GROUP_PREFIX=${KUBERNETES_CRD_GROUP_PREFIX:-}
PACKAGE_NAME=${PACKAGE_NAME:-}

print_usage() {
  echo "Usage: generate Java model classes from CRDs" >& 2
  echo " -n: the prefix of the target CRD's api group to generate." >& 2
  echo " -p: the base package name of the generated java project. " >& 2
  echo " -o: output directory of the generated java project. " >& 2
  echo " -u: url location of the YAML manifest to install CRDs to a Kubernetes cluster. " >& 2
}

while getopts 'u:n:p:o:' flag; do
  case "${flag}" in
    u) CRD_URLS+=("${OPTARG}") ;;
    n) KUBERNETES_CRD_GROUP_PREFIX="${OPTARG}" ;;
    p) PACKAGE_NAME="${OPTARG}" ;;
    o) OUTPUT_DIR="${OPTARG}" ;;
    *) print_usage
       exit 1 ;;
  esac
done

set -e

# create a KinD cluster on the host
kind create cluster

# install CRDs to the KinD cluster and dump the swagger spec
for url in "${CRD_URLS[@]}"; do
  if [[ ! -z $url ]]; then
    kubectl apply -f "$url"
  fi
done

sleep 5
kubectl get --raw="/openapi/v2" > /tmp/swagger

echo "Verifying CRD installation.."
kubectl get crd -o name \
  | while read L
    do
      if [[ $(kubectl get $L -o jsonpath='{.status.conditions[?(@.type=="NonStructuralSchema")].status}') == "True" ]]; then
        echo "$L failed publishing openapi schema because it's attached non-structral-schema condition."
        kind delete cluster
        exit 1
      fi
      if [[ $(kubectl get $L -o jsonpath='{.spec.preserveUnknownFields}') == "true" ]]; then
        echo "$L failed publishing openapi schema because it explicitly disabled unknown fields pruning."
        kind delete cluster
        exit 1
      fi
      echo "$L successfully installed"
    done

# destroy the KinD cluster
kind delete cluster

# execute the generation script
bash java-crd-cmd.sh -n "${KUBERNETES_CRD_GROUP_PREFIX}" -p "${PACKAGE_NAME}" -l 2 -o "${OUTPUT_DIR}/gen" < /tmp/swagger

# only keep the model classes
mkdir -p "${OUTPUT_DIR}/src/main/java/${PACKAGE_NAME//.//}"
cp -r "${OUTPUT_DIR}/gen/src/main/java/${PACKAGE_NAME//.//}/models" "${OUTPUT_DIR}/src/main/java/${PACKAGE_NAME//.//}"
rm -rf "${OUTPUT_DIR}/gen"/gen/openapi
k8s-triage-robot commented 3 months ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot commented 2 months ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot commented 1 month ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

k8s-ci-robot commented 1 month ago

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to [this](https://github.com/kubernetes-client/java/issues/2619#issuecomment-2143497587): >The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. > >This bot triages issues according to the following rules: >- After 90d of inactivity, `lifecycle/stale` is applied >- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied >- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed > >You can: >- Reopen this issue with `/reopen` >- Mark this issue as fresh with `/remove-lifecycle rotten` >- Offer to help out with [Issue Triage][1] > >Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community). > >/close not-planned > >[1]: https://www.kubernetes.dev/docs/guide/issue-triage/ Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
free6om commented 1 week ago

As @scprek mentioned above, the latest version of crd-model-gen image, which is v1.0.6, is still not containing the fix. And the document is still pointing to v1.0.6.

This is inconvenient for the guys who have met the same problem. Here is a quick fix:

change the image from ghcr.io/kubernetes-client/java/crd-model-gen:v1.0.6 to docker.io/apecloud/crd-model-gen:v1.0.7 .

That is, the whole cmd will be:

docker run \
  --rm \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v "$(pwd)":"$(pwd)" \
  -ti \
  --network host \
  docker.io/apecloud/crd-model-gen:v1.0.7 \
  /generate.sh \
  -u https://gist.githubusercontent.com/yue9944882/266fee8e95c2f15a93778263633e72ed/raw/be12c13379eeed13d2532cb65da61fffb19ee3e7/crontab-crd.yaml \
  -n com.example.stable \
  -p com.example.stable \
  -o "$(pwd)"

Attention: use the official image when they have updated it.