Using Examples Throws An Error

[TheDarkula]

I worked through the example in the readme, but after applying the yaml files, the cert-manager pod logs show this:

dns.go:88] "cert-manager/challenges/Present: presenting DNS01 challenge for domain" resource_name="test-example-ca-qfkmx-1016260727-3774297136" resource_namespace="default" resource_kind="Challenge" resource_version="v1" dnsName="www.example.com" type="DNS-01" resource_name="test-example-ca-qfkmx-1016260727-3774297136" resource_namespace="default" resource_kind="Challenge" resource_version="v1" domain="www.example.com"

controller.go:167] "cert-manager/challenges: re-queuing item due to error processing" err="failed initializing powerdns provider: failed parsing provider config: error decoding solver config: illegal base64 data at input byte 6" key="default/test-example-ca-qfkmx-1016260727-3774297136

[zachomedia]

Hello @TheDarkula, if you copied the example as-is, then the caBundle value will be invalid (it expects a base64 encoded CA certificate). The example should probably leave that commented-out by default to prevent this.

If you don't need to use a custom CA Cert, then you can just comment out that setting.

[TheDarkula]

@zachomedia Ah, I understand, thank you!

Commenting that out did lead to another error, though:

1 controller.go:167] "cert-manager/challenges: re-queuing item due to error processing" err="failed loading existing records for _acme-challenge.example.com. in domain example.com.: failed loading zone example.com.: Get \"http://internal.svc.cluster.local:8081/api/v1/servers/localhost/zones/example.com.\": net/http: invalid header field value for \"X-Api-Key\"" key="default/test-example-ca-5mdzv-1016260727-365178633"

[zachomedia]

I would check that the secret containing the API key is correctly configured.

1. Check the value in the Secret object
2. Check the apiKeySecretRef points to the right secret and key

[TheDarkula]

For the secret, I did echo "api_key_here" | base64, and replaced APIKEY_BASE64 below:

apiVersion: v1
kind: Secret
metadata:
  name: pdns-api-key
  namespace: default
type: Opaque
data:
  key: APIKEY_BASE64

In the issuer, I have this:

              apiKeySecretRef:
                name: pdns-api-key
                namespace: default
                key: key

[zachomedia]

Try echo -n "API KEY", I'm guessing there is a new line at the end of the API key

[TheDarkula]

Great catch! All is well now :)

[TheDarkula]

@zachomedia I hit a bit of curiosity. I successfully added certificates for a handful of domains, but today, I am unable to:

controller.go:167] "cert-manager/challenges: re-queuing item due to error processing" err="zone com. may not be edited per config (allowed zones are [example.com.])" key="servicename/servicename-example-com-tls-5zszl-3352830494-2876302178"

util.go:83] "cert-manager/controller/certificaterequests-issuer-acme/handleOwnedResource: owning resource not found in cache" resource_name="servicename-example-com-tls-5zszl-3352830494" resource_namespace="servicename" resource_kind="Order" resource_version="v1" related_resource_namespace="servicename" related_resource_name="servicename-example-com-tls-5zszl" related_resource_kind="CertificateRequest"

controller.go:165] "cert-manager/orders: order in work queue no longer exists" err="order.acme.cert-manager.io \"servicename-example-com-tls-5zszl-3352830494\" not found"

util.go:83] "cert-manager/controller/orders/handleOwnedResource: owning resource not found in cache" resource_name="servicename-example-com-tls-5zszl-3352830494-2876302178" resource_namespace="servicename" resource_kind="Challenge" resource_version="v1" related_resource_namespace="servicename" related_resource_name="servicename-example-com-tls-5zszl-3352830494" related_resource_kind="Order"

sync.go:282] "cert-manager/challenges/finalizer: error cleaning up challenge" err="failed loading existing records for _acme-challenge.servicename.example.com. in domain com.: failed loading zone com.: Not Found" resource_name="servicename-example-com-tls-5zszl-3352830494-2876302178" resource_namespace="servicename" resource_kind="Challenge" resource_version="v1" dnsName="servicename.example.com" type="DNS-01"

util.go:83] "cert-manager/controller/orders/handleOwnedResource: owning resource not found in cache" resource_name="servicename-example-com-tls-5zszl-3352830494-2876302178" resource_namespace="servicename" resource_kind="Challenge" resource_version="v1" related_resource_namespace="servicename" related_resource_name="servicename-example-com-tls-5zszl-3352830494" related_resource_kind="Order"

Also, looking in the powerdns logs shows no errors.

[zachomedia]

@TheDarkula You have zone restrictions on the Issuer config (again probably should be commented out in the example) so it's refusing to set the DNS records.

Either update or remove the allowed-zones setting.

[TheDarkula]

I had that thought as well. With the allowed-zones section commented out, it shows this:

err="failed loading existing records for _acme-challenge.servicename.example.com. in domain com.: failed loading zone com.: Not Found"

The thing that is confusing is that it was working, and I did not change the ClusterIssuer.

[zachomedia]

It's weird that it's trying to update com., that would typically mean that cert-manager didn't find a SOA record for example.com.

I would debug this in cert manager since it passes on to the webhook what zone to update. https://cert-manager.io/docs/troubleshooting/acme/

[TheDarkula]

From the powerdns logs when I try to issue a certificate:

"GET /api/v1/servers/localhost/zones/com. HTTP/1.1" 404 114

[TheDarkula]

As ever, split-horizon DNS is difficult. The issue was having a wildcard A record in my recursive DNS server, which was removing the SOA record from the authoritative server. Thank you so much for the help again!

Also, thank you for your work on this project :)

[zachomedia]

I'm glad to hear you were able to sort it out!