Controller manager seems to be not working properly due to cachcing errors

/kind bug

What steps did you take and what happened:

I0823 03:05:37.226125       1 request.go:1212] Response Body: {"kind":"APIResourceList","apiVersion":"v1","groupVersion":"controlplane.cluster.x-k8s.io/v1beta2","resources":[{"name":"awsmanagedcontrolplanes","singularName":"awsmanagedcontrolplane","namespaced":true,"kind":"AWSManagedControlPlane","verbs":["delete","deletecollection","get","list","patch","create","update","watch"],"shortNames":["awsmcp"],"categories":["cluster-api"],"storageVersionHash":"WnEFh7oqH48="},{"name":"awsmanagedcontrolplanes/status","singularName":"","namespaced":true,"kind":"AWSManagedControlPlane","verbs":["get","patch","update"]},{"name":"rosacontrolplanes","singularName":"rosacontrolplane","namespaced":true,"kind":"ROSAControlPlane","verbs":["delete","deletecollection","get","list","patch","create","update","watch"],"shortNames":["rosacp"],"categories":["cluster-api"],"storageVersionHash":"qdhYg8dFBqo="},{"name":"rosacontrolplanes/status","singularName":"","namespaced":true,"kind":"ROSAControlPlane","verbs":["get","patch","update"]}]}
I0823 03:05:37.226379       1 shared_informer.go:337] stop requested
E0823 03:05:37.226393       1 kind.go:68] "controller-runtime/source/EventHandler: failed to get informer from cache" err="Timeout: failed waiting for *v1beta2.AWSManagedControlPlane Informer to sync"
I0823 03:05:37.226424       1 reflector.go:289] Starting reflector *v1beta2.AWSManagedControlPlane (9m13.30993253s) from pkg/mod/k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0823 03:05:37.226445       1 shared_informer.go:337] stop requested
I0823 03:05:37.226459       1 shared_informer.go:337] stop requested
E0823 03:05:37.226464       1 kind.go:68] "controller-runtime/source/EventHandler: failed to get informer from cache" err="Timeout: failed waiting for *v1beta2.AWSManagedControlPlane Informer to sync"
I0823 03:05:37.226448       1 reflector.go:289] Starting reflector *v1beta2.AWSManagedCluster (10m42.211609855s) from pkg/mod/k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
E0823 03:05:37.226477       1 kind.go:68] "controller-runtime/source/EventHandler: failed to get informer from cache" err="Timeout: failed waiting for *v1beta2.AWSCluster Informer to sync"
I0823 03:05:37.226481       1 reflector.go:295] Stopping reflector *v1beta2.AWSManagedCluster (10m42.211609855s) from pkg/mod/k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0823 03:05:37.226486       1 shared_informer.go:337] stop requested
I0823 03:05:37.226433       1 shared_informer.go:337] stop requested
E0823 03:05:37.226497       1 kind.go:68] "controller-runtime/source/EventHandler: failed to get informer from cache" err="Timeout: failed waiting for *v1beta2.AWSCluster Informer to sync"
I0823 03:05:37.226437       1 shared_informer.go:337] stop requested
E0823 03:05:37.226511       1 kind.go:68] "controller-runtime/source/EventHandler: failed to get informer from cache" err="Timeout: failed waiting for *v1beta1.Machine Informer to sync"
I0823 03:05:37.226449       1 reflector.go:295] Stopping reflector *v1beta2.AWSManagedControlPlane (9m13.30993253s) from pkg/mod/k8s.io/client-go@v0.28.4/tools/cache/reflector.go:229
I0823 03:05:37.226538       1 internal.go:530] "Stopping and waiting for webhooks"
E0823 03:05:37.226499       1 kind.go:68] "controller-runtime/source/EventHandler: failed to get informer from cache" err="Timeout: failed waiting for *v1beta2.AWSManagedCluster Informer to sync"
I0823 03:05:37.226429       1 shared_informer.go:337] stop requested
E0823 03:05:37.226562       1 kind.go:68] "controller-runtime/source/EventHandler: failed to get informer from cache" err="Timeout: failed waiting for *v1beta2.AWSManagedCluster Informer to sync"
I0823 03:05:37.226441       1 shared_informer.go:337] stop requested
E0823 03:05:37.226576       1 kind.go:68] "controller-runtime/source/EventHandler: failed to get informer from cache" err="Timeout: failed waiting for *v1beta2.AWSManagedControlPlane Informer to sync"
I0823 03:05:37.226596       1 server.go:249] "controller-runtime/webhook: Shutting down webhook server with timeout of 1 minute"
I0823 03:05:37.226660       1 internal.go:533] "Stopping and waiting for HTTP servers"
I0823 03:05:37.226688       1 server.go:43] "shutting down server" kind="health probe" addr="[::]:9440"
I0823 03:05:37.226717       1 internal.go:537] "Wait completed, proceeding to shutdown the manager"
E0823 03:05:37.226747       1 logger.go:99] "setup: problem running manager" err="failed to start metrics server: failed to create listener: listen tcp: address 8443: missing port in address"

Environment: Here my manifest (controller manager)

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    argocd.argoproj.io/instance: cluster-api
    cluster.x-k8s.io/provider: infrastructure-aws
    control-plane: capa-controller-manager
  name: capa-controller-manager
  namespace: capa-system
spec:
  replicas: 1
  selector:
    matchLabels:
      cluster.x-k8s.io/provider: infrastructure-aws
      control-plane: capa-controller-manager
  template:
    metadata:
      labels:
        cluster.x-k8s.io/provider: infrastructure-aws
        control-plane: capa-controller-manager
    spec:
      containers:
        - args:
            - '--leader-elect'
            - '--feature-gates=EKS=true'
            - '--v=10'
            - '--diagnostics-address=8443'
            - '--insecure-diagnostics=false'
          env:
            - name: AWS_SHARED_CREDENTIALS_FILE
              value: /home/.aws/credentials
          image: >-
            kcr.dev.kabang.cloud/container-registry/external/cluster-api/cluster-api-aws-controller:v2.4.1
          imagePullPolicy: IfNotPresent
          imagePullSecrets: kcr-token
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: healthz
            periodSeconds: 10
          name: manager
          ports:
            - containerPort: 9443
              name: webhook-server
              protocol: TCP
            - containerPort: 9440
              name: healthz
              protocol: TCP
            - containerPort: 8443
              name: metrics
              protocol: TCP
          readinessProbe:
            httpGet:
              path: /readyz
              port: healthz
          securityContext:
            allowPrivilegeEscalation: false
            capabilities:
              drop:
                - ALL
            runAsGroup: 65532
            runAsUser: 65532
          volumeMounts:
            - mountPath: /tmp/k8s-webhook-server/serving-certs
              name: cert
              readOnly: true
      securityContext:
        fsGroup: 1000
        runAsNonRoot: true
        seccompProfile:
          type: RuntimeDefault
      serviceAccountName: capa-controller-manager
      terminationGracePeriodSeconds: 10
      tolerations:
        - effect: NoSchedule
          key: node-role.kubernetes.io/master
        - effect: NoSchedule
          key: node-role.kubernetes.io/control-plane
      volumes:
        - name: cert
          secret:
            defaultMode: 420
            secretName: capa-webhook-service-cert

Cluster-api-provider-aws version: v2.4.1
Kubernetes version: (use kubectl version): 1.27 (eks)
OS (e.g. from /etc/os-release):

kubernetes-sigs / cluster-api-provider-aws

Controller manager seems to be not working properly due to cachcing errors #5103