acmesh-official / acme.sh

A pure Unix shell script implementing ACME client protocol
https://acme.sh
GNU General Public License v3.0
38.97k stars 4.94k forks source link

Error, can not get domain token entry *.domain.tld for dns-01 #4972

Open Hossy opened 8 months ago

Hossy commented 8 months ago

Steps to reproduce

Honestly, not quite sure how to get the CA stuck in this pickle, but I can tell you the symptoms. (note: strings have been randomized to look real and protect security)

[Tue Jan 30 00:45:18 CST 2024] acme.sh:issue:4671                       response='{"identifier":{"type":"dns","value":"domain.tld"},"status":"invalid","expires":"2024-02-24T07:05:50Z","challenges":[{"type":"dns-01","url":"https://acme.zerossl.com/v2/DV90/chall/Lt2qWPyV5nnHR8xZXClQHr","status":"invalid","error":{},"token":"pxFDt2QEktWOZsPIJq5r8H_cTasXmLBvUeJoQRRh4Y9D"}],"wildcard":true}#https://acme.zerossl.com/v2/DV90/authz/X8_yBrTW3uufk6DJb__mlt'

...

[Tue Jan 30 00:45:18 CST 2024] entry
[Tue Jan 30 00:45:18 CST 2024] Error, can not get domain token entry *.domain.tld for dns-01

The Problem

The problem that is happening is due to the code on line 4693: entry="$(echo "$response" | _egrep_o '[^\{]*"type":"'$vtype'"[^\}]*')"

where this is assuming the end of the challenge array item must be a } and the key:value pair of "error":{} is throwing a wrench in that.

The Solution (closer anyway, not perfect)

I have come up with a replacement for line 4693: entry="$(echo "$response" | sed -n 's#"challenges":\[\([^]]*\)\]#\1#p' | sed 's#\(\}\),\({\)#\1\n\2#' | grep '"type":"'$vtype'"')"

It isn't perfect, but without a tool like jq and to remain POSIX compliant, it is extremely difficult to cover absolutely every case. For example, if, for some reason, the challenge array items start containing arrays themselves, we'll be back in the same boat. At least this should work until then. :-)

Takalele commented 8 months ago

Hi, i'm also hitting this issue when using zerossl.com as CA. @Hossy thank you for the workaround. BR Takalele

Hossy commented 8 months ago

Hi, i'm also hitting this issue when using zerossl.com as CA. @Hossy thank you for the workaround. BR Takalele

I'll pass your thanks onto my wife. lol. I was up until 4am reading 13.5k lines of bash debugging output. #insanethings

You can pull this if you use Docker: ghcr.io/hossy/acme.sh:jhtest-noautoupdate. Just be sure you want all my changes: https://github.com/acmesh-official/acme.sh/compare/master...Hossy:acme.sh:jhtest-noautoupdate

daBee commented 8 months ago

I am getting the same for http-01. This is not my forte, so I thought I would post all the same.

Hossy commented 8 months ago

[Tue Jan 30 00:45:18 CST 2024] entry

If you're running with --debug 2 and you're seeing lines similar to the ones below, this fix might help you. Couldn't hurt to try it, but if you do, please share your results.

[Tue Jan 30 00:45:18 CST 2024] entry
[Tue Jan 30 00:45:18 CST 2024] Error, can not get domain token entry *.domain.tld for [...]
daBee commented 8 months ago
acme.sh --upgrade
acme.sh --issue -d example.com -d www.example.com -w /home/user3/sites/example/public --debug 2

[Tue Jan 30 15:25:52 EST 2024] Error, can not get domain token "type":"http-01","url":"https://acme.zerossl.com/v2/DV90/chall/pKJI6s74oF9vkw-kxJL_GQ","status":"invalid","error":{
[Tue Jan 30 15:25:52 EST 2024] pid
[Tue Jan 30 15:25:52 EST 2024] No need to restore nginx, skip.
[Tue Jan 30 15:25:52 EST 2024] _clearupdns
[Tue Jan 30 15:25:52 EST 2024] dns_entries
[Tue Jan 30 15:25:52 EST 2024] skip dns.
[Tue Jan 30 15:25:52 EST 2024] _on_issue_err
[Tue Jan 30 15:25:52 EST 2024] Please add '--debug' or '--log' to check more details.
[Tue Jan 30 15:25:52 EST 2024] See: https://github.com/acmesh-official/acme.sh/wiki/How-to-debug-acme.sh
[Tue Jan 30 15:25:52 EST 2024] _chk_vlist
[Tue Jan 30 15:25:52 EST 2024] Diagnosis versions:
openssl:openssl
OpenSSL 3.0.7 1 Nov 2022 (Library: OpenSSL 3.0.7 1 Nov 2022)
apache:
apache doesn't exist.
nginx:
nginx version: nginx/1.22.1
built by gcc 11.3.1 20221121 (Red Hat 11.3.1-4) (GCC)
built with OpenSSL 3.0.7 1 Nov 2022
TLS SNI support enabled
configure arguments: --prefix=/usr/share/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --http-client-body-temp-path=/var/lib/nginx/tmp/client_body --http-proxy-temp-path=/var/lib/nginx/tmp/proxy --http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi --http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi --http-scgi-temp-path=/var/lib/nginx/tmp/scgi --pid-path=/run/nginx.pid --lock-path=/run/lock/subsys/nginx --user=nginx --group=nginx --with-compat --with-debug --with-file-aio --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_degradation_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_image_filter_module=dynamic --with-http_mp4_module --with-http_perl_module=dynamic --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-http_xslt_module=dynamic --with-mail=dynamic --with-mail_ssl_module --with-openssl-opt=enable-ktls --with-pcre --with-pcre-jit --with-stream=dynamic --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-threads --with-cc-opt='-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -march=x86-64-v2 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection' --with-ld-opt='-Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,-E'
socat:
socat by Gerhard Rieger and contributors - see www.dest-unreach.org
socat version 1.7.4.1 on Aug 10 2021 00:00:00
   running on Linux version #1 SMP PREEMPT_DYNAMIC Tue Aug 8 04:07:27 EDT 2023, release 5.14.0-284.25.1.el9_2.x86_64, machine x86_64
features:
  #define WITH_STDIO 1
  #define WITH_FDNUM 1
  #define WITH_FILE 1
  #define WITH_CREAT 1
  #define WITH_GOPEN 1
  #define WITH_TERMIOS 1
  #define WITH_PIPE 1
  #define WITH_UNIX 1
  #define WITH_ABSTRACT_UNIXSOCKET 1
  #define WITH_IP4 1
  #define WITH_IP6 1
  #define WITH_RAWIP 1
  #define WITH_GENERICSOCKET 1
  #define WITH_INTERFACE 1
  #define WITH_TCP 1
  #define WITH_UDP 1
  #define WITH_SCTP 1
  #define WITH_LISTEN 1
  #define WITH_SOCKS4 1
  #define WITH_SOCKS4A 1
  #define WITH_VSOCK 1
  #define WITH_PROXY 1
  #define WITH_SYSTEM 1
  #define WITH_EXEC 1
  #define WITH_READLINE 1
  #define WITH_TUN 1
  #define WITH_PTY 1
  #define WITH_OPENSSL 1
  #undef WITH_FIPS
  #undef WITH_LIBWRAP
  #define WITH_SYCLS 1
  #define WITH_FILAN 1
  #define WITH_RETRY 1
  #define WITH_MSGLEVEL 0 /*debug*/
Hossy commented 8 months ago

@daBee So, --upgrade won't get you this fix. You'll either need to manually edit acme.sh, grab it off my branch, or something like that. Try replacing that line and retry your --issue command.

EDIT: But thanks. You made me look and I found a second instance of that bad line and updated it in my PR #4973.

dreyTee commented 8 months ago

@daBee So, --upgrade won't get you this fix. You'll either need to manually edit acme.sh, grab it off my branch, or something like that. Try replacing that line and retry your --issue command.

EDIT: But thanks. You made me look and I found a second instance of that bad line and updated it in my PR #4973.

Thanks for fix, saved me some precious time!

daBee commented 8 months ago

OK, that worked for domain 1 and 2. I have a third domain that won't even issue.

[Tue Feb  6 09:56:39 EST 2024] get authz objec with invalid status, please try again later.
[Tue Feb  6 09:56:39 EST 2024] _authorizations_seg='https://acme.zerossl.com/v2/DV90/authz/65bT0SA33XvgrOJgtXPViA,https://acme.zerossl.com/v2/DV90/authz/i1VXtCl26I9vEvubNbvvcQ'
[Tue Feb  6 09:56:39 EST 2024] {"identifier":{"type":"dns","value":"example.com"},"status":"invalid","expires":"2024-03-06T07:47:20Z","challenges":[{"type":"http-01","url":"https://acme.zerossl.com/v2/DV90/chall/crtE5jbLzNCGstbjH4LA-Q","status":"invalid","error":{},"token":"7UlRFrMDUzKrugN8V1N4xB9ghcb2xoOXKjtFtyVi98o"}]}
[Tue Feb  6 09:56:39 EST 2024] pid

Not sure if this is related. I'm also having security issues with domain 2 after a successful installation. I've had a look and it's not clear why it's not secure. Another issue, I'm thinking, as issuance and installation have cleared.

Hossy commented 8 months ago

OK, that worked for domain 1 and 2. I have a third domain that won't even issue.

[Tue Feb  6 09:56:39 EST 2024] get authz objec with invalid status, please try again later.
[Tue Feb  6 09:56:39 EST 2024] _authorizations_seg='https://acme.zerossl.com/v2/DV90/authz/65bT0SA33XvgrOJgtXPViA,https://acme.zerossl.com/v2/DV90/authz/i1VXtCl26I9vEvubNbvvcQ'
[Tue Feb  6 09:56:39 EST 2024] {"identifier":{"type":"dns","value":"example.com"},"status":"invalid","expires":"2024-03-06T07:47:20Z","challenges":[{"type":"http-01","url":"https://acme.zerossl.com/v2/DV90/chall/crtE5jbLzNCGstbjH4LA-Q","status":"invalid","error":{},"token":"7UlRFrMDUzKrugN8V1N4xB9ghcb2xoOXKjtFtyVi98o"}]}
[Tue Feb  6 09:56:39 EST 2024] pid

Not sure if this is related. I'm also having security issues with domain 2 after a successful installation. I've had a look and it's not clear why it's not secure. Another issue, I'm thinking, as issuance and installation have cleared.

It seems to be a problem with resolving the challenge. I would post in Discussions and include a full debug log. Also, I would check your DNS to ensure the challenge entry doesn't already exist from a previously failed run. You might need to delete it manually.

daBee commented 8 months ago

OK, just to be clear, this is not dns but http. I will post in discussions.

nmartinho commented 8 months ago

Same problem here. "get authz objec with invalid status, please try again later."

I had to switch from Zerossl to Letsencrypt, and renew now is ok, without errors.

daBee commented 8 months ago

OK good to know. I was going to try switching providers today.

Hossy commented 8 months ago

@daBee @nmartinho FWIW, everything you are describing matches the symptoms I was also experiencing. In my situation, it was not the specific provider (ZeroSSL over LetsEncrypt), it was the fact that the provider had a pending and unresolved challenge. The code fix I provided should resolve the issue you're experiencing. You can manually apply the changes in PR #4973 to your local file.

daBee commented 8 months ago

I did make those changes, and only a single domain worked. I changed to letsencrypt and everything fell into place.

Hossy commented 8 months ago

@daBee Just to confirm, you changed both lines?

daBee commented 8 months ago

Yes I did. The line numbers were different, but I found them.

Hossy commented 8 months ago

The line numbers were different, but I found them.

Are you running the latest acme.sh? acme.sh --upgrade If the upgrade takes action, you'll need to update the lines again.

daBee commented 8 months ago

No you told me that --upgrade wouldn't work because it wasn't published. It was a manual change. Anyway I'm now onto the new certs from letsencrypt.

Hossy commented 8 months ago

My apologies for the confusion. I meant that --upgrade would not provide you the fix I did, but I was assuming you were already running the latest version. According to your post, the problem using ZeroSSL will self-resolve (expire) 2024-03-06T07:47:20Z, in case you run into this issue again with any provider.

daBee commented 8 months ago

OK that's how I understood your --upgrade comment. All is good. I was expecting these changes to see a new update soon.

Hossy commented 8 months ago

FWIW, you should consider doing an upgrade (then reapplying my patch). I can't find the text (or anything similar to) get authz objec with invalid status, please try again later. in the current code, so it may be that your second problem was fixed already.

daBee commented 8 months ago

I did my last --upgrade not a week ago. I patched days after you posted those line changes.

https://github.com/acmesh-official/acme.sh
v3.0.8
Hossy commented 8 months ago

Hmm, ok, well that isn't it then. Did you get a chance to post in Discussions yet? I was hoping to see your full debug log. I was also going to ask if you could try it with --debug 3 as well.

daBee commented 8 months ago

No I didn't post because the new provider worked. I had to get the certs working. BTW the reason I tripped over this is because the renew crontab wasn't working, so they expired.

Hossy commented 8 months ago

your crontab should have a call to --cron, not --renew iirc

Hossy commented 8 months ago

OK, that worked for domain 1 and 2. I have a third domain that won't even issue.

[Tue Feb  6 09:56:39 EST 2024] get authz objec with invalid status, please try again later.
[Tue Feb  6 09:56:39 EST 2024] _authorizations_seg='https://acme.zerossl.com/v2/DV90/authz/65bT0SA33XvgrOJgtXPViA,https://acme.zerossl.com/v2/DV90/authz/i1VXtCl26I9vEvubNbvvcQ'
[Tue Feb  6 09:56:39 EST 2024] {"identifier":{"type":"dns","value":"example.com"},"status":"invalid","expires":"2024-03-06T07:47:20Z","challenges":[{"type":"http-01","url":"https://acme.zerossl.com/v2/DV90/chall/crtE5jbLzNCGstbjH4LA-Q","status":"invalid","error":{},"token":"7UlRFrMDUzKrugN8V1N4xB9ghcb2xoOXKjtFtyVi98o"}]}
[Tue Feb  6 09:56:39 EST 2024] pid

Not sure if this is related. I'm also having security issues with domain 2 after a successful installation. I've had a look and it's not clear why it's not secure. Another issue, I'm thinking, as issuance and installation have cleared.

For better or worse, I am now experiencing the same problem you are/were seeing. Investigating now.

[Wed Feb 14 13:29:15 CST 2024] get authz objec with invalid status, please try again later.
[Wed Feb 14 13:29:15 CST 2024] _authorizations_seg='https://acme.zerossl.com/v2/DV90/authz/xxxxxxxxxx,https://acme.zerossl.com/v2/DV90/authz/xxxxxxxxxx'
[Wed Feb 14 13:29:15 CST 2024] {"identifier":{"type":"dns","value":"mydomain.com"},"status":"invalid","expires":"2024-03-14T18:59:20Z","challenges":[{"type":"dns-01","url":"https://acme.zerossl.com/v2/DV90/chall/xxxxxxxxxx","status":"invalid","error":{},"token":"xxxxxxxxxx"}]}
[Wed Feb 14 13:29:15 CST 2024] pid
Hossy commented 8 months ago

OK, that worked for domain 1 and 2. I have a third domain that won't even issue.

[Tue Feb  6 09:56:39 EST 2024] get authz objec with invalid status, please try again later.
[Tue Feb  6 09:56:39 EST 2024] _authorizations_seg='https://acme.zerossl.com/v2/DV90/authz/65bT0SA33XvgrOJgtXPViA,https://acme.zerossl.com/v2/DV90/authz/i1VXtCl26I9vEvubNbvvcQ'
[Tue Feb  6 09:56:39 EST 2024] {"identifier":{"type":"dns","value":"example.com"},"status":"invalid","expires":"2024-03-06T07:47:20Z","challenges":[{"type":"http-01","url":"https://acme.zerossl.com/v2/DV90/chall/crtE5jbLzNCGstbjH4LA-Q","status":"invalid","error":{},"token":"7UlRFrMDUzKrugN8V1N4xB9ghcb2xoOXKjtFtyVi98o"}]}
[Tue Feb  6 09:56:39 EST 2024] pid

Not sure if this is related. I'm also having security issues with domain 2 after a successful installation. I've had a look and it's not clear why it's not secure. Another issue, I'm thinking, as issuance and installation have cleared.

For better or worse, I am now experiencing the same problem you are/were seeing. Investigating now.

[Wed Feb 14 13:29:15 CST 2024] get authz objec with invalid status, please try again later.
[Wed Feb 14 13:29:15 CST 2024] _authorizations_seg='https://acme.zerossl.com/v2/DV90/authz/xxxxxxxxxx,https://acme.zerossl.com/v2/DV90/authz/xxxxxxxxxx'
[Wed Feb 14 13:29:15 CST 2024] {"identifier":{"type":"dns","value":"mydomain.com"},"status":"invalid","expires":"2024-03-14T18:59:20Z","challenges":[{"type":"dns-01","url":"https://acme.zerossl.com/v2/DV90/chall/xxxxxxxxxx","status":"invalid","error":{},"token":"xxxxxxxxxx"}]}
[Wed Feb 14 13:29:15 CST 2024] pid

I solved my problem. Two things were going on... 1) I had changed my DNS provider for the domain being renewed and that change was not yet reflected in the config file (most likely due to the second issue); 2) my script I run to call --issue was passing --keylength and --always-force-new-domain-key after each domain (-d domain.com) parameter and this somehow pissed acme.sh off. I didn't bother to figure out why it was pissed off, I just removed the duplicate parameters being passed and the issue went away.

@daBee I suspect ultimately the problem was due to the DNS provider change. So, double-check your *.conf file and if the DNS provider is wrong, fix it by calling --issue --dns

daBee commented 8 months ago

I wasn't using DNS for the test. This was all http/https.

anjia0532 commented 7 months ago

https://zerossl.com/pricing/

image

Hossy commented 7 months ago

https://zerossl.com/pricing/

image

That is only for certificates created without ACME. You 100% can do wildcard ACME certificates on the Free plan.