search

hashicorp / vault

A tool for secrets management, encryption as a service, and privileged access management
https://www.vaultproject.io/
Other
31.16k stars 4.21k forks source link

DNS misbehaving with ACME dns-01 validation #25993

Open micheelengronne opened 7 months ago

micheelengronne commented 7 months ago

Describe the bug I set up a Vault cluster with and ACME endpoint in a podman container with dual stack IPv6 enabled.

I created a TXT record for the DN I want and nslookup inside the container detects it correctly.

~ $ nslookup -q=txt _acme-challenge.wg-1.undercloud.coppint.test
Server:     10.89.1.1
Address:    10.89.1.1:53

Non-authoritative answer:
_acme-challenge.wg-1.undercloud.coppint.test    text = "a_cwDX44Jcv2_MLsMwvRac7KsTdaaMFWuxjOSNxbGIg"
_acme-challenge.wg-1.undercloud.coppint.test    text = "IAaGBJusgJlM9pMMZmxf8fkAdUxBZg-kNdgJYnsCK9o"
_acme-challenge.wg-1.undercloud.coppint.test    text = "WgINYO6IZGqKawU8cVWdbXH2cYk_LrN60DSkIdaQvEA"

Vault fails:

[ERROR] secrets.pki.pki_0ad38abd: ACME validation failed for 9ebf92bf-4373-1920-4ed3-feb19bdff641-dns-01: removing challenge validation attempt and not retrying 9ebf92bf-4373-1920-4ed3-feb19bdff641-dns-01; previous error: reached max error attempts for challenge 9ebf92bf-4373-1920-4ed3-feb19bdff641-dns-01: Response received didn't match the challenge's requirements: error validating dns-01 challenge 9ebf92bf-4373-1920-4ed3-feb19bdff641-dns-01: dns-01: failed to lookup TXT records for domain (_acme-challenge.wg-1.undercloud.coppint.test) via resolver [fc00:fc00:fc01:fc01:fc00:fc00:fc00:1]:50053: lookup _acme-challenge.wg-1.undercloud.coppint.test on [fdfd:5033:251b:e4f2::1]:53: server misbehaving; this may occur if the validation target was misconfigured: check that challenge responses are available at the required locations and retry.

The /etc/resolv.conf inside the container:

search dns.podman
nameserver 10.89.1.1
nameserver fdfd:5033:251b:e4f2::1

To Reproduce Steps to reproduce the behavior:

  1. Start Vault with a PKI engine and ACME inside podman
  2. Ask the ACME endpoint to validate via dns-01 the record you want
  3. See error

Expected behavior I expect the TXT record to be correctly detected.

Environment: Official Vault Docker image: hashicorp/vault:1.14.9@sha256:5918004894d96d5edac4522cee0d45d394b61882916a6854d3c7d1fdb82d506d

stevendpclark commented 7 months ago

Hello @micheelengronne,

Could you have overridden the dns_resolver value within Vault's ACME configuration? That error message is confusing at best, as we seem to have configured the dialer for [fc00:fc00:fc01:fc01:fc00:fc00:fc00:1]:50053 but the Go error, I believe, is referencing [fdfd:5033:251b:e4f2::1]:53: 🤔

micheelengronne commented 7 months ago

Hello @stevendpclark yes I did. But I tested without overriding and had the same result.

micheelengronne commented 7 months ago

The fc0 dns is part of the dns_resolver value. The fdfd dns is set by the podman network my container is in.