search

joohoi / acme-dns

Limited DNS server with RESTful HTTP API to handle ACME DNS challenges easily and securely.
MIT License
2.13k stars 233 forks source link

ACME-DNS-API not pulling a certificate for itself #324

Open protogenxl opened 1 year ago

protogenxl commented 1 year ago

Hi I have been working on setting up a acme-dns and have ran into an issue where the web API is not pulling it's own let's encrypt cert. I feel like I am missing something simple but I am to far in to see what is behind me.

Any help would be appreciated

halibut.juggedfish.com will be playing the part of my public dns record and 8.8.8.8 will be playing the part of my public ip address

ERROR

root@halibut ~]# systemctl status acme-dns -l
? acme-dns.service - Limited DNS server with RESTful HTTP API to handle ACME DNS challenges easily and securely
   Loaded: loaded (/etc/systemd/system/acme-dns.service; enabled; vendor preset: disabled)
   Active: active (running) since Thu 2022-12-22 09:49:24 EST; 46min ago
 Main PID: 737 (acme-dns)
   CGroup: /system.slice/acme-dns.service
           +-737 /usr/local/bin/acme-dns

Dec 22 09:49:24 halibut.juggedfish.com systemd[1]: Started Limited DNS server with RESTful HTTP API to handle ACME DNS challenges easily and securely.
Dec 22 09:49:24 halibut.juggedfish.com acme-dns[737]: time="2022-12-22T09:49:24-05:00" level=info msg="Using config file" file=/etc/acme-dns/config.cfg
Dec 22 09:49:48 halibut.juggedfish.com acme-dns[737]: time="2022-12-22T09:49:48-05:00" level=info msg="http: TLS handshake error from 10.248.2.1:55191: no certificate available for 'halibut.juggedfish.com'"
Dec 22 09:49:48 halibut.juggedfish.com acme-dns[737]: time="2022-12-22T09:49:48-05:00" level=info msg="http: TLS handshake error from 10.248.2.1:55192: no certificate available for 'halibut.juggedfish.com'"
Dec 22 09:49:48 halibut.juggedfish.com acme-dns[737]: time="2022-12-22T09:49:48-05:00" level=info msg="http: TLS handshake error from 10.248.2.1:55193: no certificate available for 'halibut.juggedfish.com'"
Dec 22 09:49:48 halibut.juggedfish.com acme-dns[737]: time="2022-12-22T09:49:48-05:00" level=info msg="http: TLS handshake error from 10.248.2.1:55194: no certificate available for 'halibut.juggedfish.com'"

CONFIG

# ================= DO NOT MODIFY THIS FILE =================
#
# Manual changes will be lost when this file is regenerated.
#
# Please read the developer's guide, which is available
# at NethServer official site: https://www.nethserver.org
#
#

[general]
# dns interface
listen = "10.110.144.53:1053"
# protocol, "udp", "udp4", "udp6" or "tcp", "tcp4", "tcp6"
protocol = "both"
# domain name to serve the requests off of
domain = "halibut.juggedfish.com"
# zone name server
nsname = "ns1.halibut.juggedfish.com"
# admin email address, where @ is substituted with .
nsadmin = "halibut@juggedfish.com"
# predefined records served in addition to the TXT
records = [
    # default A
    "halibut.juggedfish.com. A 8.8.8.8",
    # A
    "ns1.halibut.juggedfish.com. A 8.8.8.8",
    "ns2.halibut.juggedfish.com. A 8.8.8.8",
    # NS
    "halibut.juggedfish.com. NS ns1.halibut.juggedfish.com.",
    "halibut.juggedfish.com. NS ns2.halibut.juggedfish.com.",
]
# debug messages from CORS etc
debug = true

[database]
# Database engine to use, sqlite3 or postgres
engine = "sqlite3"
connection = "/etc/acme-dns/acme-dns.db"

[api]
# listen ip eg. 127.0.0.1
ip = "10.110.144.53"
# disable registration endpoint
disable_registration = false
# listen port, eg. 443 for default HTTPS
port = "443"
# possible values: "letsencrypt", "letsencryptstaging", "cert", "none"
tls = "letsencrypt"
# only used if tls = "letsencrypt"
acme_cache_dir = "/etc/acme-dns/api-certs"
# CORS AllowOrigins, wildcards can be used
corsorigins = [
    "*"
]
# use HTTP header to get the client ip
use_header = false
# header name to pull the ip address / list of ip addresses from
header_name = "X-Forwarded-For"

[logconfig]
# logging level: "error", "warning", "info" or "debug"
loglevel = "warning"
# possible values: stdout, TODO file & integrations
logtype = "stdout"
# file path for logfile TODO
# logfile = "./acme-dns.log"
# format, either "json" or "text"
logformat = "text"

PUBLIC DNS


A
ns1.halibut
8.8.8.8
15 minutes

A
ns2.halibut
8.8.8.8
15 minutes

CNAME
_acme-challenge.halibut
halibut.juggedfish.com
15 minutes

NS
halibut
ns2.halibut.juggedfish.com
15 minutes

NS
halibut
ns1.halibut.juggedfish.com
15 minutes
webprofusion-chrisc commented 1 year ago

Are port 53 DNS queries against your instance working? You are listening on 1053 but will need to port forward this externally from 53 for normal DNS queries to work (you may already be doing that). I'd imagine if that doesn't work then it also won't be able to use itself to complete a DNS challenge for it's own cert. There was/used to be an http-01 challenge mode but I don't know the config to use that instead of DNS validation.

novakele commented 1 year ago

Hi @protogenxl, I ran into a similar issue.

In my case, I was running the acme-dns.service as a non-root user, and the user did not have write permission in his home directory. By default, the service uses WorkingDirectory=~.

Are you running the service as root?

EDIT Does the user runner the service has write permissions in /etc/acme-dns/api-certs?

protogenxl commented 1 year ago

@webprofusion-chrisc yes the DNS forward on my firewall seems to working correctly

2022-12-27 08:11:12 Allow 8.0.38.4 8.8.8.8 dns/udp 53597 53 6-WAN 1-Trusted ProxyAllow: DNS question match   (LetsEncryptDNS-proxy-00) DNS-Incoming.1 proc_id="dns-proxy" rc="590" msg_id="1DFF-000E" proxy_act="DNS-Incoming.1" rule_name="*" query_type="A" question="nS2.halibut.juggedfish.com" | Traffic

2022-12-27 08:11:12 Allow 8.0.38.4 8.8.8.8 dns/udp 53597 53 6-WAN 1-Trusted Allowed 82 55 (LetsEncryptDNS-proxy-00)  proc_id="firewall" rc="100" msg_id="3000-0148" dst_ip_nat="10.110.144.53" dst_port_nat="1053" | Traffic

@novakele I am running the service as acme-dns and permissions appear to be correct

[root@halibut ~]# cd /etc/acme-dns/api-certs
[root@halibut api-certs]# ls -lah
total 0
drwx------ 4  992 acme-dns 100 Dec 27 08:03 .
drwxrwxr-x 3 root acme-dns  60 Dec 21 06:17 ..
drwx------ 3  992 acme-dns  60 Dec 20 23:53 acme
drwx------ 2  992 acme-dns 142 Dec 27 08:03 locks
-rw------- 1  992 acme-dns   0 Dec 21 06:17 rw_test_288443979776093768
-rw------- 1  992 acme-dns   0 Dec 21 06:17 rw_test_9142020181166123590
novakele commented 1 year ago

It is strange that the owner uid (992) does not resolve to the user acme-dns.

Could you provide the output of id acme-dns? Should the uid of acme-dns be anything else than 992, that is your problem.

Here are the permissions for my instance:

root@lighthouse:~# tree -pufidg /var/lib/acme-dns/
[drwxr-xr-x acme-dns acme-dns]  /var/lib/acme-dns
[drwx------ acme-dns acme-dns]  /var/lib/acme-dns/api-certs
[drwx------ acme-dns acme-dns]  /var/lib/acme-dns/api-certs/acme
[drwx------ acme-dns acme-dns]  /var/lib/acme-dns/api-certs/acme/acme-v02.api.letsencrypt.org-directory
[drwx------ acme-dns acme-dns]  /var/lib/acme-dns/api-certs/acme/acme-v02.api.letsencrypt.org-directory/users
[drwx------ acme-dns acme-dns]  /var/lib/acme-dns/api-certs/acme/acme-v02.api.letsencrypt.org-directory/users/<EMAIL>
[drwx------ acme-dns acme-dns]  /var/lib/acme-dns/api-certs/certificates
[drwx------ acme-dns acme-dns]  /var/lib/acme-dns/api-certs/certificates/acme-v02.api.letsencrypt.org-directory
[drwx------ acme-dns acme-dns]  /var/lib/acme-dns/api-certs/certificates/acme-v02.api.letsencrypt.org-directory/<DOMAIN>
[drwx------ acme-dns acme-dns]  /var/lib/acme-dns/api-certs/locks
[drwx------ acme-dns acme-dns]  /var/lib/acme-dns/api-certs/ocsp

Also, here is the output of the same commands you ran (I use /var/lib/acme-dns instead of /etc/acme-dns for the home directory):

root@lighthouse:~# cd /var/lib/acme-dns/api-certs/

root@lighthouse:/var/lib/acme-dns/api-certs# ls -lah
total 24K
drwx------ 6 acme-dns acme-dns 4.0K Dec 23 22:19 .
drwxr-xr-x 3 acme-dns acme-dns 4.0K Dec 23 23:31 ..
drwx------ 3 acme-dns acme-dns 4.0K Dec 23 22:19 acme
drwx------ 3 acme-dns acme-dns 4.0K Dec 23 22:19 certificates
drwx------ 2 acme-dns acme-dns 4.0K Dec 23 22:19 locks
drwx------ 2 acme-dns acme-dns 4.0K Dec 23 22:19 ocsp
p3l1 commented 1 year ago

See #315, I encountered similar problems. I am using the Dockerfile to run acme-dns. The v0.8 release works just fine. In my opinion this is not an environment problem, but a problem of the software itself.

On my profile I've got an improved Dockerfile based on the v0.8 release.

jeffsf commented 1 year ago

I'm seeing similar behavior to what has been reported here and in #315 on a new installation in FreeBSD 13.1-RELEASE-p5 and the current upstream compiled and packaged acme-dns-1.0_3,1 (installed today).

time="2023-02-01T19:13:53-08:00" level=info msg="Using config file" file=/usr/local/etc/acme-dns/config.cfg
time="2023-02-01T19:13:53-08:00" level=info msg="Connected to database"
time="2023-02-01T19:13:53-08:00" level=debug msg="Adding new record to domain" domain=<DELEGATED_NS_NAME>. recordtype=A
time="2023-02-01T19:13:53-08:00" level=debug msg="Adding new record to domain" domain=<DELEGATED_NS_NAME>. recordtype=NS
time="2023-02-01T19:13:53-08:00" level=debug msg="Adding new record to domain" domain=<DELEGATED_NS_NAME>. recordtype=SOA
time="2023-02-01T19:13:53-08:00" level=info msg="Listening HTTPS" domain=<DELEGATED_NS_NAME> host="0.0.0.0:443"
time="2023-02-01T19:13:53-08:00" level=info msg="Listening DNS" addr= proto=tcp
time="2023-02-01T19:13:53-08:00" level=info msg="Listening DNS" addr= proto=udp
time="2023-02-01T19:14:36-08:00" level=debug msg="Answering question for domain" domain=should.fail.<MY_DOMAIN>. qtype=A rcode=NXDOMAIN
time="2023-02-01T19:15:11-08:00" level=info msg="http: TLS handshake error from <TEST_HOST_IP>:57801: no certificate available for '<DELEGATED_NS_NAME>'"

On a subsequent restart, I additionally get

time="2023-02-01T19:29:33-08:00" level=info msg="2023/02/01 19:29:33 [INFO][FileStorage:/var/db/acme-dns/api-certs] Lock for 'issue_cert_<DELEGATED_NS_NAME>' is stale (created: 2023-02-01 19:13:53.163243238 -0800 PST, last update: 2023-02-01 19:29:23.133675565 -0800 PST); removing then retrying: /var/db/acme-dns/api-certs/locks/issue_cert_<DELEGATED_NS_NAME>.lock"
sudo find var/db/acme-dns/api-certs/ -type d -exec ls -ld {} \;
drwx------  4 root  wheel  4 Feb  1 19:13 var/db/acme-dns/api-certs/
drwx------  2 root  wheel  3 Feb  1 19:13 var/db/acme-dns/api-certs/locks
drwx------  4 root  wheel  4 Feb  1 19:15 var/db/acme-dns/api-certs/acme
drwx------  3 root  wheel  3 Feb  1 19:13 var/db/acme-dns/api-certs/acme/acme-v02.api.letsencrypt.org-directory
drwx------  3 root  wheel  3 Feb  1 19:13 var/db/acme-dns/api-certs/acme/acme-v02.api.letsencrypt.org-directory/users
drwx------  2 root  wheel  4 Feb  1 19:13 var/db/acme-dns/api-certs/acme/acme-v02.api.letsencrypt.org-directory/users/<NOTIFICATON_EMAIL>
drwx------  3 root  wheel  3 Feb  1 19:15 var/db/acme-dns/api-certs/acme/acme-staging-v02.api.letsencrypt.org-directory
drwx------  3 root  wheel  3 Feb  1 19:15 var/db/acme-dns/api-certs/acme/acme-staging-v02.api.letsencrypt.org-directory/users
drwx------  2 root  wheel  4 Feb  1 19:15 var/db/acme-dns/api-certs/acme/acme-staging-v02.api.letsencrypt.org-directory/users/<NOTIFICATON_EMAIL>
$ egrep -v '^#' usr/local/etc/acme-dns/config.cfg
[general]
protocol = "both"
domain = "<DELEGATED_NS_NAME>"
nsname = "<DELEGATED_NS_NAME>"
nsadmin = "<HOSTMASTER@DOMAIN.NAME>"
records = [
    # domain pointing to the public IP of your acme-dns server
    "<DELEGATED_NS_NAME>. A <DELEGATED_NS_PUBLIC_IP>",
    # specify that auth.example.org will resolve any *.auth.example.org records
    "<DELEGATED_NS_NAME>. NS <DELEGATED_NS_NAME>.",
]
debug = true

[database]
engine = "sqlite3"
connection = "/var/db/acme-dns/acme-dns.db"

[api]
ip = "0.0.0.0"
disable_registration = false
port = "443"
tls = "letsencrypt"
acme_cache_dir = "/var/db/acme-dns/api-certs"
notification_email = "<NOTIFICATON_EMAIL>"
corsorigins = [
    "*"
]
use_header = false
header_name = "X-Forwarded-For"

[logconfig]
loglevel = "debug"
logtype = "stdout"
logformat = "text"
jeffsf commented 9 months ago

Confirmed still an issue in FreeBSD 13.2 and package acme-dns-1.0_12,1

No changes in behavior identified

tigeli commented 7 months ago

Might be related to this https://github.com/joohoi/acme-dns/issues/339#issuecomment-1935928848

jeffsf commented 7 months ago

I did try adding the CAA record as described on #339 with no change in behavior

Removing the entire /var/db/acme-dns/api-certs directory and allowing it to be recreated also did not change the behavior.

It appears that the query is coming in for the TXT record. The following appears to repeat periodically.

time="2024-02-11T17:41:11-08:00" level=debug msg="Answering question for domain" domain=<DELEGATED_NS_NAME>. qtype=AAAA rcode=NOERROR
time="2024-02-11T17:41:11-08:00" level=debug msg="Answering question for domain" domain=_Acme-cHALlENgE.<DELEGATED_NS_NAME>. qtype=TXT rcode=NOERROR
time="2024-02-11T17:41:11-08:00" level=debug msg="Answering question for domain" domain=<DELEGATED_NS_NAME>. qtype=AAAA rcode=NOERROR
time="2024-02-11T17:41:11-08:00" level=debug msg="Answering question for domain" domain=_AcMe-chaLLEnGe.<DELEGATED_NS_NAME>. qtype=TXT rcode=NOERROR
time="2024-02-11T17:41:11-08:00" level=debug msg="Answering question for domain" domain=<DELEGATED_NS_NAME>. qtype=AAAA rcode=NOERROR
time="2024-02-11T17:41:11-08:00" level=debug msg="Answering question for domain" domain=<f4...UUID...b1>.<DELEGATED_NS_NAME>. qtype=TXT rcode=NOERROR
time="2024-02-11T17:41:11-08:00" level=debug msg="Answering question for domain" domain=<DELEGATED_NS_NAME>. qtype=AAAA rcode=NOERROR
time="2024-02-11T17:41:11-08:00" level=debug msg="Answering question for domain" domain=_aCME-CHALlENgE.<DELEGATED_NS_NAME>. qtype=TXT rcode=NOERROR
time="2024-02-11T17:41:11-08:00" level=debug msg="Answering question for domain" domain=<DELEGATED_NS_NAME>. qtype=AAAA rcode=NOERROR
time="2024-02-11T17:41:11-08:00" level=debug msg="Answering question for domain" domain=<f4...UUID...b1>.<DELEGATED_NS_NAME>. qtype=TXT rcode=NOERROR
time="2024-02-11T17:41:11-08:00" level=debug msg="Answering question for domain" domain=<DELEGATED_NS_NAME>. qtype=AAAA rcode=NOERROR
time="2024-02-11T17:41:11-08:00" level=debug msg="Answering question for domain" domain=<f4...UUID...b1>.<DELEGATED_NS_NAME>. qtype=TXT rcode=NOERROR

The requests for the TXT record come in from a variety of IP addresses, including one that reverse-resolves to outbound1g.letsencrypt.org and another to outbound2e.letsencrypt.org

There is no certificate present anywhere under /var/db/acme-dns/api-certs and openssl s_client confirms that there is no certificate available.

Edit: For clarity, the instance works through the API to have other clients renew certificates. It just isn't able to take care of its own.