Certificate provisioning stuck on FAILED_NOT_VISIBLE

[adrianchifor]

I got the controller to see the ingress annotations and it issued a few certificates, however they are stuck in FAILED_NOT_VISIBLE for a few hours now. The DNS is controlled in the same GCP project by Cloud DNS and is resolvable and reachable publicly so I'm not sure what the issue might be. Any extra information that might help?

[adrianchifor]

Didn't realize that I had to attach the provisioning certificates to the HTTPS load balancer for them to be become verified, I thought GCP would verify them based on the DNS pointing to the LB alone. The controller should automatically attach the certs to the LB frontend right?

[g00nix]

I have a certificate that got properly attached to a load balancer, and the DNS is pointing to the LB address, however the certificate is still stuck in FAILED_NOT_VISIBLE.

[tapvirvirk]

I am facing same issue. Any luck what it is stuck at this state(FAILED_NOT_VISIBLE)?

[g00nix]

I am facing same issue. Any luck what it is stuck at this state(FAILED_NOT_VISIBLE)?

It gets stuck some times. If you have the DNS and load balancer set up correctly, it should work. If it doesn't, try deleting and the certificate and adding it back.

[tapvirvirk]

I tried re-creating the certificate but still it is failing with the same reason. I verified the DNS using nslookup & ping. It is getting resolved with the load balancer front end ip. Regarding Load Balancer, I have tested using http protocal and with self signed certificate. I'm able to hit the application successfully. Specifically, any other thing that I need to check in the DNS or load balancer config?

[adrianchifor]

@tapvirvirk did you attach the certificate to a load balancer https forward rule?

[tapvirvirk]

@adrianchifor - yes I did. I'm creating the certificate from the load balancer frontend console

[adrianchifor]

That's strange. Keep in mind it might take between 10min-2h to provision the cert. As long as DNS for that endpoint is pointing to the HTTPS LB and the created certificate is attached to a rule, it should be ok.

[tapvirvirk]

I just re-created the cert. It always throws an error after 19 mins. "domainStatus": { "xxxxxx.com": "FAILED_NOT_VISIBLE" }

[adrianchifor]

You sure your DNS zone is public? I can't resolve that domain.

[adrianchifor]

$ dig ministopstores.com

; <<>> DiG 9.10.6 <<>> ministopstores.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 16081
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;ministopstores.com.        IN  A

;; Query time: 49 msec
;; SERVER: 2401:fa00:fa::1#53(2401:fa00:fa::1)
;; WHEN: Fri Mar 22 15:10:16 GMT 2019
;; MSG SIZE  rcvd: 47

[tapvirvirk]

I created the entry yesterday around 4 PM EST. Is it because of the propogation time?

[tapvirvirk]

I managed to resolve it. It was the wrong DNSSEC entries in the domain registration.

[ajainvivek]

Do we need to create certificate manually via LoadBalancer or does this service create a certificate which we need to attach to the LoadBalancer

GKE managed certificate yaml

apiVersion: networking.gke.io/v1beta1
kind: ManagedCertificate
metadata:
  name: example-certificate
spec:
  domains:
    - example.com

Ingress

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: "example-ingress"
  annotations:
    kubernetes.io/ingress.global-static-ip-name: "example-static-ip"
    kubernetes.io/ingress.allow-http: "true"
    networking.gke.io/managed-certificates: "example-certificate"
spec:
  rules:
  - http:
      paths:
      - path: /*
        backend:
          serviceName: "example-service"
          servicePort: 80

[ajainvivek]

I had to manually create a certificate, I am not sure if I am doing it right

[michelau]

It worked for me today to just specify the Google Managed Cert as a "pre-shared cert" as shown below:

ingress.gcp.kubernetes.io/pre-shared-cert: "fitpeak-app-alpha"

[hbksilver]

I have the same issue. It's going for 2 weeks now. I gues gcp is suck

[wolmi]

I was able to fix it by removing the ManagedCertificate object and the annotation on the ingress and then applying both again, it forces the process to restart.