search

acmesh-official / acme.sh

A pure Unix shell script implementing ACME client protocol
https://acme.sh
GNU General Public License v3.0
38.32k stars 4.89k forks source link

Strange resolving issues with curl in acme.sh #1396

Open Kedarius opened 6 years ago

Kedarius commented 6 years ago

I am having strange issues with CURL in acme.sh. First I thought that it is some network configuration issue (and it probably is) but acme.sh works fine with --use-wget and CURL itself works fine too... System is Fedora 27, curl is curl-7.55.1-9.fc27.x86_64 and acme.sh is downloaded today (16 mar 2018).

The problem is that sometimes curl downloads go to our main webserver instead of LE's servers. It is possible if the resolving is working wrong or the hostname is bad - we unfortunately use wildcard DNS name for our main webserver *.ourcompany.com - and the fedora installer defaults to use searchdomain in resolv.conf. So it may happen that if the acme.api.letsencrypt.org is not found, the client may try acme.api....ourcompany.com and it returns the wildcarded address of our main server. However the resolving seems to work fine - nslookup, host and ping do resolve correctly and curl from command line goes to right server to... The log in case of the problem looks like this:

Fri Mar 16 13:12:25 CET 2018] tigger domain validation.
[Fri Mar 16 13:12:25 CET 2018] _t_url='https://acme-staging.api.letsencrypt.org/acme/challenge/zOBQ9NnpoHg9m2xRi44JLQXfjU0F3yFKESVrvslshng/109746118'
[Fri Mar 16 13:12:25 CET 2018] _t_key_authz='Pb7nrAPudUpAAQnXO_JT9m2njt9iYTvAH71_1dYzgBI.KI9i8dKQrWfa3mbFufG0O5udTshEUN9GEfzYn0x99UU'
[Fri Mar 16 13:12:25 CET 2018] url='https://acme-staging.api.letsencrypt.org/acme/challenge/zOBQ9NnpoHg9m2xRi44JLQXfjU0F3yFKESVrvslshng/109746118'
[Fri Mar 16 13:12:25 CET 2018] payload='{"resource": "challenge", "keyAuthorization": "Pb7nrAPudUpAAQnXO_JT9m2njt9iYTvAH71_1dYzgBI.KI9i8dKQrWfa3mbFufG0O5udTshEUN9GEfzYn0x99UU"}'
[Fri Mar 16 13:12:25 CET 2018] Use cached jwk for file: /root/.acme.sh/ca/acme-staging.api.letsencrypt.org/account.key
[Fri Mar 16 13:12:25 CET 2018] Use _CACHED_NONCE='XZdhC6eEWax0AtpztLiIF06sVKyuKEwtShpJRNSBnPc'
[Fri Mar 16 13:12:25 CET 2018] nonce='XZdhC6eEWax0AtpztLiIF06sVKyuKEwtShpJRNSBnPc'
[Fri Mar 16 13:12:25 CET 2018] POST
[Fri Mar 16 13:12:25 CET 2018] _post_url='https://acme-staging.api.letsencrypt.org/acme/challenge/zOBQ9NnpoHg9m2xRi44JLQXfjU0F3yFKESVrvslshng/109746118'
[Fri Mar 16 13:12:25 CET 2018] body='{"header": {"alg": "RS256", "jwk": {"e": "AQAB", "kty": "RSA", "n": "k........9", "payload": "e.....Q", "signature": "U......w"}'
[Fri Mar 16 13:12:25 CET 2018] _CURL='curl -L --silent --dump-header /root/.acme.sh/http.header  --trace-ascii /tmp/tmp.FGarFfUCDn  -g  -H "Content-Type: application/jose+json" '
[Fri Mar 16 13:12:26 CET 2018] _ret='0'
[Fri Mar 16 13:12:26 CET 2018] original='<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
<head>
<title>Object not found!</title>
<link rev="made" href="mailto:webmaster@ourcompany.com" />
</head>

<body>
<h1>Object not found!</h1>
<p>
    The requested URL was not found on this server.
......

In the other try everything went OK until the final certificate download. Again acme.sh downloaded "certificate" but it was the error page from our server and OpenSSL was not able to parse the certificate. I did download the mentioned URL afterwards with command-line curl and it downloaded the certificate without any issues. (I do not have the logs from this at hand)

I tried to use different DNS server (8.8.8.8), remove the searchdomain option, even putting the hostname into /etc/hosts.

For now it seems to be solved by using wget but I would like to find out what the problem is. Is there any way how to debug CURL more or is there any difference how acme.sh uses CURL (compared to command line)?

Neilpang commented 6 years ago

show me the full log with --debug 2

Kedarius commented 6 years ago

Here is the full log problem.txt the problem seems to be around the line 269, where acme.sh tried to download the certificate and clearly goes to our server and then to the LE server - according to headers and the response. If you try to decode the base64 response you will see that its concatenated our not found page and the certificate.... When I download it afterwards manually everything works fine:

[root@smtp-out ~]# curl https://acme-staging.api.letsencrypt.org/acme/cert/fa1092468d732f5c14d20b0f84307e86a438 --output - | openssl x509 -inform der -noout -text
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1534  100  1534    0     0   1534      0  0:00:01 --:--:--  0:00:01  6972
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            fa:10:92:46:8d:73:2f:5c:14:d2:0b:0f:84:30:7e:86:a4:38
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN = Fake LE Intermediate X1
        Validity
            Not Before: Mar 18 14:53:34 2018 GMT
            Not After : Jun 16 14:53:34 2018 GMT
        Subject: CN = smtp-out2.cybersales.cz
Neilpang commented 6 years ago

try with the latest code please:

acme.sh --upgrade
Kedarius commented 6 years ago

It seems to be working now. At least I did manage to issue a certificate (on staging API) 3 times with CURL.... seemsok.txt