stolostron / deploy

Deploy Development Builds of Open Cluster Management (OCM) on RedHat Openshift Container Platform
Apache License 2.0
158 stars 153 forks source link

[BUG][DEPLOY] Unable to cleanup Klusterlet CRD #179

Closed minsikl closed 2 years ago

minsikl commented 3 years ago

Describe the bug A clear and concise description of what the bug is.

After the execution of clean-clusters.sh, klusterlets.operator.open-cluster-management.io has still existed.

To Reproduce Steps to reproduce the behavior:

  1. Execute clean-clusters.sh
  2. Generate an import code and execute it on the target cluster.

Expected behavior A clear and concise description of what you expected to happen. The cluster is successfully imported.

Screenshots If applicable, add screenshots to help explain your problem.

Error from server (AlreadyExists): error when creating "STDIN": customresourcedefinitions.apiextensions.k8s.io "klusterlets.operator.open-cluster-management.io" already exists
The cluster cannot be imported because its Klusterlet CRD already exists.
Either the cluster was already imported, or it was not detached completely during a previous detach process.
Detach the existing cluster before trying the import again.%

Desktop (please complete the following information):

Additional context Add any other context about the problem here.

wdrdres3qew5ts21 commented 3 years ago

I confirm this was bug if Hub is down or had been destroy but somehow we didn't detach Managed Cluster it will make Klusterlet stuck forever. In the meanwhile if someone encounter this problem you need go Custom Resource Definition then copy whole of Klusterlet then using oc replace -f original-klusterlet.yaml then it will delete both CRD and Klusterlet object instance. Tested on Openshift 4.7 and using ACM 2.3 original-klusterlet.yaml

kind: CustomResourceDefinition
apiVersion: apiextensions.k8s.io/v1
metadata:
  name: klusterlets.operator.open-cluster-management.io
spec:
  group: operator.open-cluster-management.io
  names:
    plural: klusterlets
    singular: klusterlet
    kind: Klusterlet
    listKind: KlusterletList
  scope: Cluster
  versions:
    - name: v1
      served: true
      storage: true
      schema:
        openAPIV3Schema:
          description: >-
            Klusterlet represents controllers on the managed cluster. When
            configured, the Klusterlet requires a secret named of
            bootstrap-hub-kubeconfig in the same namespace to allow API requests
            to the hub for the registration protocol.
          type: object
          properties:
            apiVersion:
              description: >-
                APIVersion defines the versioned schema of this representation
                of an object. Servers should convert recognized schemas to the
                latest internal value, and may reject unrecognized values. More
                info:
                https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
              type: string
            kind:
              description: >-
                Kind is a string value representing the REST resource this
                object represents. Servers may infer this from the endpoint the
                client submits requests to. Cannot be updated. In CamelCase.
                More info:
                https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
              type: string
            metadata:
              type: object
            spec:
              description: >-
                Spec represents the desired deployment configuration of
                Klusterlet agent.
              type: object
              properties:
                clusterName:
                  description: >-
                    ClusterName is the name of the managed cluster to be created
                    on hub. The Klusterlet agent generates a random name if it
                    is not set, or discovers the appropriate cluster name on
                    openshift.
                  type: string
                externalServerURLs:
                  description: >-
                    ExternalServerURLs represents the a list of apiserver urls
                    and ca bundles that is accessible externally If it is set
                    empty, managed cluster has no externally accessible url that
                    hub cluster can visit.
                  type: array
                  items:
                    description: >-
                      ServerURL represents the apiserver url and ca bundle that
                      is accessible externally
                    type: object
                    properties:
                      caBundle:
                        description: >-
                          CABundle is the ca bundle to connect to apiserver of
                          the managed cluster. System certs are used if it is
                          not set.
                        type: string
                        format: byte
                      url:
                        description: >-
                          URL is the url of apiserver endpoint of the managed
                          cluster.
                        type: string
                namespace:
                  description: >-
                    Namespace is the namespace to deploy the agent. The
                    namespace must have a prefix of "open-cluster-management-",
                    and if it is not set, the namespace of
                    "open-cluster-management-agent" is used to deploy agent.
                  type: string
                registrationImagePullSpec:
                  description: >-
                    RegistrationImagePullSpec represents the desired image
                    configuration of registration agent.
                  type: string
                workImagePullSpec:
                  description: >-
                    WorkImagePullSpec represents the desired image configuration
                    of work agent.
                  type: string
            status:
              description: Status represents the current status of Klusterlet agent.
              type: object
              properties:
                conditions:
                  description: >-
                    Conditions contain the different condition statuses for this
                    Klusterlet. Valid condition types are: Applied: components
                    have been applied in the managed cluster. Available:
                    components in the managed cluster are available and ready to
                    serve. Progressing: components in the managed cluster are in
                    a transitioning state. Degraded: components in the managed
                    cluster do not match the desired configuration and only
                    provide degraded service.
                  type: array
                  items:
                    description: StatusCondition contains condition information.
                    type: object
                    properties:
                      lastTransitionTime:
                        description: >-
                          LastTransitionTime is the last time the condition
                          changed from one status to another.
                        type: string
                        format: date-time
                      message:
                        description: >-
                          Message is a human-readable message indicating details
                          about the last status change.
                        type: string
                      reason:
                        description: >-
                          Reason is a (brief) reason for the condition's last
                          status change.
                        type: string
                      status:
                        description: >-
                          Status is the status of the condition. One of True,
                          False, Unknown.
                        type: string
                      type:
                        description: Type is the type of the cluster condition.
                        type: string
      subresources:
        status: {}
  conversion:
    strategy: None
  preserveUnknownFields: true
yylin1 commented 2 years ago

I meet the same issue, through using oc replace -f original-klusterlet.yaml can solve the problem by @wdrdres3qew5ts21.

dmaneru commented 2 years ago

Hi! Thanks for the help. This is still usefull with ACM Version 2.5.1 and an Openshift Dedicated Cluster in v4.8