search

GoogleCloudPlatform / gke-managed-certs

Managed Certificates for Kubernetes clusters using GCLB
Apache License 2.0
246 stars 32 forks source link

FORBIDDEN error after certificate creation #43

Closed lwsanty closed 3 years ago

lwsanty commented 4 years ago

I'm creating ingresses with managed certificates as in example https://cloud.google.com/kubernetes-engine/docs/how-to/managed-certs

I even still have one running on subdomain1.domain.com I also have been successfully creating ingresses for the other sub-domains, but today I faced this problem.

kubectl describe managedcertificate -n web-app
Name:         web-app-certificate
Namespace:    web-app
Labels:       <none>
Annotations:  <none>
API Version:  networking.gke.io/v1beta1
Kind:         ManagedCertificate
Metadata:
  Creation Timestamp:  2020-01-13T19:39:37Z
  Generation:          2
  Resource Version:    2270
  Self Link:           /apis/networking.gke.io/v1beta1/namespaces/web-app/managedcertificates/web-app-certificate
  UID:                 6ea7a4bd-363c-11ea-840c-42010af00146
Spec:
  Domains:
// here's 
    subdomain2.domain.com
Status:
  Certificate Name:  mcrt-cfb380b2-0b2c-4deb-b264-1e5be4ad259a
  Domain Status:
Events:
  Type     Reason        Age                   From                            Message
  ----     ------        ----                  ----                            -------
  Warning  BackendError  6m9s                  managed-certificate-controller  operation operation-1578944378860-59c0aa2d274a8-547f28d8-6dacdff0 failed: FORBIDDEN
  Warning  BackendError  5m58s                 managed-certificate-controller  operation operation-1578944390237-59c0aa3800bc9-25ad682d-099f8de1 failed: FORBIDDEN
  Warning  BackendError  5m47s                 managed-certificate-controller  operation operation-1578944401176-59c0aa426f7b3-13685221-86c3432c failed: FORBIDDEN
  Warning  BackendError  5m44s                 managed-certificate-controller  operation operation-1578944404387-59c0aa457f52d-456340a9-ecc77fa4 failed: FORBIDDEN
  Warning  BackendError  5m36s                 managed-certificate-controller  operation operation-1578944412291-59c0aa4d092b9-f667224d-1470767b failed: FORBIDDEN
  Warning  BackendError  5m24s                 managed-certificate-controller  operation operation-1578944424029-59c0aa583ad65-b073f0c1-a547e6a6 failed: FORBIDDEN
  Warning  BackendError  5m13s                 managed-certificate-controller  operation operation-1578944435216-59c0aa62e6263-3a24c18d-fe24c347 failed: FORBIDDEN
  Warning  BackendError  5m1s                  managed-certificate-controller  operation operation-1578944446746-59c0aa6de4dbb-bb645422-cdeb522c failed: FORBIDDEN
  Warning  BackendError  4m49s                 managed-certificate-controller  operation operation-1578944458846-59c0aa796f19f-4fd9164f-c53f59d8 failed: FORBIDDEN
  Warning  BackendError  16s (x18 over 4m36s)  managed-certificate-controller  (combined from similar events): operation operation-1578944731319-59c0ab7d48df7-733c60e6-604c77bd failed: FORBIDDEN

Is there any chance to know more details beyond the FORBIDDEN?

victorboissiere commented 4 years ago

I have the exact same issue on a managed GKE cluster v1.14.8-gke.33

drcca commented 4 years ago

I've run into this before and the problem was that I ran into a QUOTA limit. Check SSL Cert quotas?

lwsanty commented 4 years ago

@drcca thank you for a reply! Indeed back in the days when I submitted this issue the reason was in quotas eventually.

However, I have a doubt about closing this particular issue, because essentially it could be any other error and this commands' output does not provide any valuable insights.

krzykwas commented 4 years ago

Thanks for reporting this issue. The out-of-quota condition should be handled properly. I haven't yet had time to investigate it, but I have it on my list. Sorry I can't promise any time to take a look at it.

krzykwas commented 4 years ago

The bug is fixed in the newest release, v1.0.0 (not yet released in GKE).

KudMath commented 4 years ago

@krzykwas is there a way to bypass the issue in the meantime?