search

qqshfox / cert-manager-webhook-dnspod

DNSPod Webhook for Cert Manager
Apache License 2.0
55 stars 34 forks source link

does it post the request to solverName.groupName to access api ? #6

Closed TonyLuo closed 3 years ago

TonyLuo commented 4 years ago

I got this error when set groupName and solverName as following. any idea how to fix it?

groupName: cert-manager.io
solverName: dnspod
E0711 17:25:57.464031       1 controller.go:143] cert-manager/controller/challenges "msg"="re-queuing item  due to error processing" "error"="the server is currently unable to handle the request (post dnspod.cert-manager.io)" 
helm version
version.BuildInfo{Version:"v3.2.4", GitCommit:"0ad800ef43d3b826f31a5ad8dfbb4fe05d143688", GitTreeState:"clean", GoVersion:"go1.13.12"}

helm install cert-manager-webhook-dnspod --namespace cert-manager \
   ./cert-manager-webhook-dnspod/deploy/cert-manager-webhook-dnspod \
   --set groupName=cert-manager.io \
   --set secrets.apiID="myapiID",secrets.apiToken="myapiToken" \
   --set clusterIssuer.enabled=true,clusterIssuer.email="myemail@gmail.com"

letsencrypt-dnspod-staging-issuer.yaml

apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
  name: letsencrypt-dnspod-staging
spec:
  acme:
    email: myemail@gmail.com
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-staging-dnspod
    solvers:
    - dns01:
        webhook:
          groupName: cert-manager.io
          solverName: dnspod
          config:
            apiID: "my-apiID",
            apiTokenSecretRef:
              key: api-token
              name: cert-manager-webhook-dnspod-secret

certificate.yaml 

apiVersion: cert-manager.io/v1alpha2
kind: Certificate
metadata:
  name: demo-tls
spec:
  secretName: demo-tls
  commonName: "xxxx.cn"
  dnsNames:
  - "xxxx.cn"
  - "*.xxxx.cn"
  issuerRef:
    name: letsencrypt-dnspod-staging
    kind: ClusterIssuer
qqshfox commented 4 years ago

@TonyLuo I don't think cert-manager will post anything to solverName.groupName. After searching "the server is currently unable to handle the request" on Google, I found this error comes from metrics-server. I guess it might be something wrong with the cert-manager-webhook-dnspod pod which couldn't response the post request from metrics-server.

TonyLuo commented 4 years ago

I haven't installed metrics-server on my k8s cluster. is metrics-server mandatory for cert-manager-webhook-dnspod?

TonyLuo commented 4 years ago

after installing metric-server, still got the same error message: kubectl top nodes NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
master1 118m 2% 2058Mi 26%

kubectl logs cert-manager-8494747bb6-595bv -n cert-manager | less

E0712 16:24:10.144895 1 controller.go:143] cert-manager/controller/challenges "msg"="re-queuing item due to error processing" "error"="the server is currently unable to handle the request (post dnspod.cert-manager.io)"

qqshfox commented 4 years ago

You're absolutely right! This error has nothing to do with metrics-server. Found actual error message here in k8s/k8s.

It looks like the api-server complained about the cert-manager-webhook-dnspod svc returned 503 for a POST request. Please check the logs from cert-manager-webhook-dnspod pod.

TonyLuo commented 4 years ago

FYI

kubectl logs cert-manager-webhook-dnspod-94647b479-n4wj8 -n cert-manager 

W0713 15:20:01.126853       1 configmap_cafile_content.go:102] unable to load initial CA bundle for: "client-ca::kube-system::extension-apiserver-authentication::client-ca-file" due to: configmap "extension-apiserver-authentication" not found
W0713 15:20:01.126931       1 configmap_cafile_content.go:102] unable to load initial CA bundle for: "client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" due to: configmap "extension-apiserver-authentication" not found
I0713 15:20:01.136576       1 configmap_cafile_content.go:205] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0713 15:20:01.136578       1 configmap_cafile_content.go:205] Starting client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
I0713 15:20:01.136597       1 shared_informer.go:197] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
I0713 15:20:01.136597       1 shared_informer.go:197] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0713 15:20:01.136660       1 dynamic_serving_content.go:129] Starting serving-cert::/tls/tls.crt::/tls/tls.key
I0713 15:20:01.137016       1 secure_serving.go:178] Serving securely on [::]:443
I0713 15:20:01.137169       1 tlsconfig.go:219] Starting DynamicServingCertificateController
I0713 15:20:01.236694       1 shared_informer.go:204] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file 
I0713 15:20:01.236897       1 shared_informer.go:204] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file

kubectl logs -n cert-manager cert-manager-8494747bb6-kmmrs

I0713 15:24:25.793584       1 dns.go:92] cert-manager/controller/challenges/Present "msg"="presenting DNS01 challenge for domain" "dnsName"="xxx.cn" "domain"="xxxx.cn" "resource_kind"="Challenge" "resource_name"="xxxx-tls-628110643-4267525627-3257411312" "resource_namespace"="default" "type"="dns-01" 
E0713 15:24:25.794363       1 controller.go:143] cert-manager/controller/challenges "msg"="re-queuing item  due to error processing" "error"="the server is currently unable to handle the request (post dnspod.cert-manager.io)" "key"="default/xxxx-tls-628110643-4267525627-3257411312" 
I0713 15:24:45.794523       1 controller.go:141] cert-manager/controller/challenges "msg"="syncing item" "key"="default/xxx-tls-628110643-4267525627-3257411312" 
I0713 15:24:45.794725       1 dns.go:92] cert-manager/controller/challenges/Present "msg"="presenting DNS01 challenge for domain" "dnsName"="xxxx.cn" "domain"="xxxx.cn" "resource_kind"="Challenge" "resource_name"="xxxx-tls-628110643-4267525627-3257411312" "resource_namespace"="default" "type"="dns-01" 
E0713 15:24:45.795982       1 controller.go:143] cert-manager/controller/challenges "msg"="re-queuing item  due to error processing" "error"="the server is currently unable to handle the request (post dnspod.cert-manager.io)" "key"="default/xxxx-tls-628110643-4267525627-3257411312" 
I0713 15:25:25.796144       1 controller.go:141] cert-manager/controller/challenges "msg"="syncing item" "key"="default/xxxx-tls-628110643-4267525627-3257411312" 
I0713 15:25:25.796310       1 dns.go:92] cert-manager/controller/challenges/Present "msg"="presenting DNS01 challenge for domain" "dnsName"="xxxx.cn" "domain"="xxxx.cn" "resource_kind"="Challenge" "resource_name"="xxx-tls-628110643-4267525627-3257411312" "resource_namespace"="default" "type"="dns-01" 
E0713 15:25:25.797144       1 controller.go:143] cert-manager/controller/challenges "msg"="re-queuing item  due to error processing" "error"="the server is currently unable to handle the request (post dnspod.cert-manager.io)" "key"="default/xxx-tls-628110643-4267525627-3257411312"
qqshfox commented 4 years ago

Have you tried to change groupName anything else rather than cert-manager.io? There might a conflict with other api resources of cert-manager.

TonyLuo commented 4 years ago

tried to change groupName=cert-manager-dnspod, still got the same error

kubectl logs -n cert-manager cert-manager-8494747bb6-kmmrs

I0713 17:02:16.068131       1 controller.go:141] cert-manager/controller/challenges "msg"="syncing item" "key"="default/xxxx-tls-628110643-4267525627-1478355005" 
I0713 17:02:16.068200       1 logger.go:99] Calling GetChallenge
I0713 17:02:17.925637       1 controller.go:141] cert-manager/controller/orders "msg"="syncing item" "key"="default/xxxx-tls-628110643-4267525627" 
I0713 17:02:17.925840       1 logger.go:149] Calling DNS01ChallengeRecord
I0713 17:02:17.925892       1 sync.go:179] cert-manager/controller/orders "msg"="No action taken" "resource_kind"="Order" "resource_name"="xxxx-tls-628110643-4267525627" "resource_namespace"="default" 
I0713 17:02:17.925906       1 controller.go:147] cert-manager/controller/orders "msg"="finished processing work item" "key"="default/xxxx-tls-628110643-4267525627" 
I0713 17:02:17.926070       1 controller.go:147] cert-manager/controller/challenges "msg"="finished processing work item" "key"="default/xxxx-tls-628110643-4267525627-1478355005" 
I0713 17:02:17.926090       1 controller.go:141] cert-manager/controller/challenges "msg"="syncing item" "key"="default/xxxx-tls-628110643-4267525627-1478355005" 
I0713 17:02:17.926215       1 dns.go:92] cert-manager/controller/challenges/Present "msg"="presenting DNS01 challenge for domain" "dnsName"="xxxx.cn" "domain"="xxxx.cn" "resource_kind"="Challenge" "resource_name"="xxxx-tls-628110643-4267525627-1478355005" "resource_namespace"="default" "type"="dns-01" 
E0713 17:02:17.932335       1 controller.go:143] cert-manager/controller/challenges "msg"="re-queuing item  due to error processing" "error"="the server is currently unable to handle the request (post dnspod.cert-manager-dnspod)" "key"="default/xxxx-tls-628110643-4267525627-1478355005" 
I0713 17:02:17.932545       1 controller.go:141] cert-manager/controller/orders "msg"="syncing item" "key"="default/xxxx-tls-628110643-4267525627" 
I0713 17:02:17.932652       1 logger.go:149] Calling DNS01ChallengeRecord
I0713 17:02:17.932742       1 controller.go:141] cert-manager/controller/challenges "msg"="syncing item" "key"="default/xxxx-tls-628110643-4267525627-1478355005" 
I0713 17:02:17.932852       1 dns.go:92] cert-manager/controller/challenges/Present "msg"="presenting DNS01 challenge for domain" "dnsName"="xxxx.cn" "domain"="xxxx.cn" "resource_kind"="Challenge" "resource_name"="xxxx-tls-628110643-4267525627-1478355005" "resource_namespace"="default" "type"="dns-01" 
I0713 17:02:17.933082       1 sync.go:179] cert-manager/controller/orders "msg"="No action taken" "resource_kind"="Order" "resource_name"="xxxx-tls-628110643-4267525627" "resource_namespace"="default" 
I0713 17:02:17.933099       1 controller.go:147] cert-manager/controller/orders "msg"="finished processing work item" "key"="default/xxxx-tls-628110643-4267525627" 
E0713 17:02:17.933397       1 controller.go:143] cert-manager/controller/challenges "msg"="re-queuing item  due to error processing" "error"="the server is currently unable to handle the request (post dnspod.cert-manager-dnspod)" "key"="default/xxxx-tls-628110643-4267525627-1478355005" 
I0713 17:02:18.740088       1 controller.go:141] cert-manager/controller/certificates "msg"="syncing item" "key"="default/xxxx-tls" 
I0713 17:02:18.740307       1 sync.go:386] cert-manager/controller/certificates "msg"="validating existing CSR data" "related_resource_kind"="CertificateRequest" "related_resource_name"="xxxx-tls-628110643" "related_resource_namespace"="default" "resource_kind"="Certificate" "resource_name"="xxxx-tls" "resource_namespace"="default" 
I0713 17:02:18.740397       1 sync.go:511] cert-manager/controller/certificates "msg"="CertificateRequest is not in a final state, waiting until CertificateRequest is complete" "related_resource_kind"="CertificateRequest" "related_resource_name"="xxxx-tls-628110643" "related_resource_namespace"="default" "resource_kind"="Certificate" "resource_name"="xxxx-tls" "resource_namespace"="default" "state"="Pending"
I0713 17:02:18.740511       1 controller.go:147] cert-manager/controller/certificates "msg"="finished processing work item" "key"="default/xxxx-tls" 
I0713 17:02:18.749145       1 controller.go:141] cert-manager/controller/certificaterequests-issuer-acme "msg"="syncing item" "key"="default/xxxx-tls-628110643" 
I0713 17:02:18.749405       1 acme.go:201] cert-manager/controller/certificaterequests-issuer-acme/sign "msg"="acme Order resource is not in a ready state, waiting..." "related_resource_kind"="Order" "related_resource_name"="xxxx-tls-628110643-4267525627" "related_resource_namespace"="default" "resource_kind"="CertificateRequest" "resource_name"="xxxx-tls-628110643" "resource_namespace"="default" 
I0713 17:02:18.749449       1 controller.go:147] cert-manager/controller/certificaterequests-issuer-acme "msg"="finished processing work item" "key"="default/xxxx-tls-628110643" 
I0713 17:02:22.932482       1 controller.go:141] cert-manager/controller/challenges "msg"="syncing item" "key"="default/xxxx-tls-628110643-4267525627-1478355005" 
I0713 17:02:22.932657       1 dns.go:92] cert-manager/controller/challenges/Present "msg"="presenting DNS01 challenge for domain" "dnsName"="xxxx.cn" "domain"="xxxx.cn" "resource_kind"="Challenge" "resource_name"="xxxx-tls-628110643-4267525627-1478355005" "resource_namespace"="default" "type"="dns-01" 
E0713 17:02:22.933457       1 controller.go:143] cert-manager/controller/challenges "msg"="re-queuing item  due to error processing" "error"="the server is currently unable to handle the request (post dnspod.cert-manager-dnspod)" "key"="default/xxxx-tls-628110643-4267525627-1478355005" 
I0713 17:02:42.933610       1 controller.go:141] cert-manager/controller/challenges "msg"="syncing item" "key"="default/xxxx-tls-628110643-4267525627-1478355005" 
I0713 17:02:42.933806       1 dns.go:92] cert-manager/controller/challenges/Present "msg"="presenting DNS01 challenge for domain" "dnsName"="xxxx.cn" "domain"="xxxx.cn" "resource_kind"="Challenge" "resource_name"="xxxx-tls-628110643-4267525627-1478355005" "resource_namespace"="default" "type"="dns-01" 
E0713 17:02:42.934590       1 controller.go:143] cert-manager/controller/challenges "msg"="re-queuing item  due to error processing" "error"="the server is currently unable to handle the request (post dnspod.cert-manager-dnspod)" "key"="default/xxxx-tls-628110643-4267525627-1478355005" 
I0713 17:03:22.934744       1 controller.go:141] cert-manager/controller/challenges "msg"="syncing item" "key"="default/xxxx-tls-628110643-4267525627-1478355005" 
I0713 17:03:22.934899       1 dns.go:92] cert-manager/controller/challenges/Present "msg"="presenting DNS01 challenge for domain" "dnsName"="xxxx.cn" "domain"="xxxx.cn" "resource_kind"="Challenge" "resource_name"="xxxx-tls-628110643-4267525627-1478355005" "resource_namespace"="default" "type"="dns-01" 
E0713 17:03:22.935695       1 controller.go:143] cert-manager/controller/challenges "msg"="re-queuing item  due to error processing" "error"="the server is currently unable to handle the request (post dnspod.cert-manager-dnspod)" "key"="default/xxxx-tls-628110643-4267525627-1478355005"
qqshfox commented 4 years ago

Interesting... Could you please create a minimal script to reproduce this problem?

qqshfox commented 3 years ago

Closing this for inactivity. Free free to reopen.