No solvers available for remaining challenges

[hongyi-zhao]

On Ubuntu 20.04.3 LTS, I compiled the latest git master version of boringproxy, and have completed the following domain configuration:

    bp.hyddns.xyz - the fully qualified domain name (FQDN) of the host where boringproxy will be running.
    *.bp.hyddns.xyz - a wildcard subdomain for effortless domain mapping of clients

Then I try to start the boringproxy server as follows but failed:

$ boringproxy server -admin-domain bp.hyddns.xyz
2021/12/17 15:21:10 Starting up
2021/12/17 15:21:24 bp.hyddns.xyz: obtaining certificate: [bp.hyddns.xyz] Obtain: [bp.hyddns.xyz] solving challenges: bp.hyddns.xyz: no solvers available for remaining challenges (configured=[tls-alpn-01] offered=[http-01 dns-01 tls-alpn-01] remaining=[http-01 dns-01]) (order=https://acme-v02.api.letsencrypt.org/acme/order/324767840/47937107570) (ca=https://acme-v02.api.letsencrypt.org/directory)

Any hints for solving this problem?

Regards, HZ

[anderspitman]

This typically happens when your server isn't available over port 443, usually due to a firewall rule or not having port forwarding properly set up. Please confirm that port 443 is properly forwarded to boringproxy server.

FYI the next release of boringproxy will automatically detect if ports 80/443 aren't externally accessible and print an error/warning.

[hongyi-zhao]

I've opened up the port 443 on my firewall, but still meet the same error:

$ boringproxy server -admin-domain bp.hyddns.xyz
2021/12/18 08:22:47 Starting up
2021/12/18 08:23:01 bp.hyddns.xyz: obtaining certificate: [bp.hyddns.xyz] Obtain: [bp.hyddns.xyz] solving challenges: bp.hyddns.xyz: no solvers available for remaining challenges (configured=[tls-alpn-01] offered=[http-01 dns-01 tls-alpn-01] remaining=[http-01 dns-01]) (order=https://acme-v02.api.letsencrypt.org/acme/order/324767840/48118073280) (ca=https://acme-v02.api.letsencrypt.org/directory)

BTW, during the connecting progress, I observed the following information:

$ sudo lsof -i :443
COMMAND       PID   USER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
dnsproxy     7320 werner   11u  IPv4 3680921      0t0  TCP X10DAi-00:34502->dns.adguard.com:https (SYN_SENT)
dnsproxy     7320 werner   23u  IPv4 3680920      0t0  TCP X10DAi-00:34484->dns.adguard.com:https (SYN_SENT)
dnsproxy     7320 werner   46u  IPv4 3742622      0t0  TCP X10DAi-00:34560->dns.adguard.com:https (SYN_SENT)
dnsproxy     7320 werner   52u  IPv4 3660741      0t0  TCP X10DAi-00:34564->dns.adguard.com:https (SYN_SENT)
dnsproxy     7320 werner   60u  IPv4 3799782      0t0  TCP X10DAi-00:34600->dns.adguard.com:https (SYN_SENT)
dnsproxy     7320 werner   67u  IPv4 3809411      0t0  TCP X10DAi-00:34604->dns.adguard.com:https (SYN_SENT)
dnsproxy     7320 werner   71u  IPv4 3817556      0t0  TCP X10DAi-00:34680->dns.adguard.com:https (SYN_SENT)
dnsproxy     7320 werner   74u  IPv4 3742628      0t0  TCP X10DAi-00:34684->dns.adguard.com:https (SYN_SENT)
dnsproxy     7320 werner   76u  IPv4 3817558      0t0  TCP X10DAi-00:34756->dns.adguard.com:https (SYN_SENT)
dnsproxy     7320 werner   79u  IPv4 3641033      0t0  TCP X10DAi-00:34760->dns.adguard.com:https (SYN_SENT)
dnsproxy     7320 werner   81u  IPv4 3817566      0t0  TCP X10DAi-00:34818->dns.adguard.com:https (SYN_SENT)
dnsproxy     7320 werner   84u  IPv4 3800641      0t0  TCP X10DAi-00:34824->dns.adguard.com:https (SYN_SENT)
boringpro 2348745 werner    7u  IPv4 3783751      0t0  TCP X10DAi-00:59090->172.65.32.248:https (ESTABLISHED)

[anderspitman]

And you have a DNS A record pointed at the IP address? This error has come up before and so far I don't think boringproxy has ever been the issue.

[hongyi-zhao]

And you have a DNS A record pointed at the IP address?

I don't know if I understand what you mean, so I'd like to provide the following for your reference:

$ dig +short bp.hyddns.xyz 
www.hyddns.xyz.
106.9.183.31

[anderspitman]

Yeah so assuming your server is running on 106.9.183.31 with ports 80 and 443 open to the outside world, I don't know what's wrong. I'd recommend running something like caddy on the server and verifying it works as expected.

[hongyi-zhao]

with ports 80 and 443 open to the outside world

Must the ports 80 and 443 be open to the outside world at the same time?

I'd recommend running something like caddy on the server and verifying it works as expected.

If I understand you correctly, you mean I should first check if the web service running on port 80 and 443 on the server works as expected.

[anderspitman]

Must the ports 80 and 443 be open to the outside world at the same time? The current version might work with just 443. The next version will require both to be open, unless there's some reason people don't want that, but I'd need an explanation.

If I understand you correctly, you mean I should first check if the web service running on port 80 and 443 on the server works as expected. Yep. You could also just do a sudo python3 -m http.server 80 to test port 80 and open it in your browser. 443 can be trickier since browsers expect TLS on that port.

[hongyi-zhao]

You could also just do a sudo python3 -m http.server 80 to test port 80 and open it in your browser.

Successfully passed the test:

$ sudo python3 -m http.server 80
$ curl -I http://bp.hyddns.xyz/
HTTP/1.0 200 OK
Server: SimpleHTTP/0.6 Python/3.8.10
Date: Sat, 18 Dec 2021 07:01:52 GMT
Content-type: text/html; charset=utf-8
Content-Length: 5679

443 can be trickier since browsers expect TLS on that port.

Successfully passed the test too:

$ sudo python3 simple-https-python-server.py
$ curl -Ik https://bp.hyddns.xyz:443
HTTP/1.0 200 OK
Server: SimpleHTTP/0.6 Python/3.8.10
Date: Sat, 18 Dec 2021 07:32:24 GMT
Content-type: text/html; charset=utf-8
Content-Length: 5641
Access-Control-Allow-Origin: *
Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corp

The simple-https-python-server.py is adapted from here:

#https://gist.github.com/dergachev/7028596#gistcomment-3943886
#https://gist.github.com/Alexufo/2303bff77f0a16ba83568f0260b8cf47
#$ sudo python3 simple-https-python-server.py
# Test it with the following method:
#$ curl -Ik https://bp.hyddns.xyz:443

import http.server
import socket
import ssl
import os

os.system("openssl req -nodes -x509 -newkey rsa:4096 -keyout localhost.pem -out localhost.pem -days 365 -subj '/CN=localhost'")
server_address = ('0.0.0.0', 443)

hostname = socket.gethostname()
local_ip = socket.gethostbyname(hostname)

print("Open https://localhost:443")
print('Open https://' + local_ip + ':443')

#os.chdir("./dist/")

class CORSHTTPRequestHandler(http.server.SimpleHTTPRequestHandler):
    extensions_map = {
        '': 'application/octet-stream',
        '.manifest': 'text/cache-manifest',
        '.html': 'text/html',
        '.png': 'image/png',
        '.jpg': 'image/jpg',
        '.svg': 'image/svg+xml',
        '.css': 'text/css',
        '.js': 'application/x-javascript',
        '.wasm': 'application/wasm',
        '.json': 'application/json',
        '.xml': 'application/xml',
    }

    def end_headers(self):
        # Include additional response headers here. CORS for example:
        self.send_header('Access-Control-Allow-Origin', '*')
        self.send_header('Cross-Origin-Opener-Policy', 'same-origin')
        self.send_header('Cross-Origin-Embedder-Policy', 'require-corp')
        http.server.SimpleHTTPRequestHandler.end_headers(self)

httpd = http.server.HTTPServer(server_address, CORSHTTPRequestHandler)
ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
ctx.check_hostname = False
ctx.load_cert_chain(certfile='localhost.pem')  # with key inside
httpd.socket = ctx.wrap_socket(httpd.socket, server_side=True)
httpd.serve_forever()

OTOH, I try to get Let's Encrypt HTTPS Cert for Server, but failed:

$ sudo apt install certbot
$ sudo certbot certonly --register-unsafely-without-email --standalone -d bp.hyddns.xyz
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator standalone, Installer None
Registering without email!

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Please read the Terms of Service at
https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf. You must
agree in order to register with the ACME server at
https://acme-v02.api.letsencrypt.org/directory
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(A)gree/(C)ancel: a
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for bp.hyddns.xyz
Waiting for verification...
Challenge failed for domain bp.hyddns.xyz
http-01 challenge for bp.hyddns.xyz
Cleaning up challenges
Some challenges have failed.

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: bp.hyddns.xyz
   Type:   connection
   Detail: Fetching
   http://bp.hyddns.xyz/.well-known/acme-challenge/Hiff0CcOlyj2F0D6qgcknnJGy3YTqOnLZS2AZLAi4ms:
   Timeout during connect (likely firewall problem)

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address. Additionally, please check that
   your computer has a publicly routable IP address and that no
   firewalls are preventing the server from communicating with the
   client. If you're using the webroot plugin, you should also verify
   that you are serving files from the webroot path you provided.
 - Your account credentials have been saved in your Certbot
   configuration directory at /etc/letsencrypt. You should make a
   secure backup of this folder now. This configuration directory will
   also contain certificates and private keys obtained by Certbot so
   making regular backups of this folder is ideal.

I also installed the latest git master version of certbot by the method given here. But it still failed and encountered the following error:

$ which certbot 
/usr/local/bin/certbot
$ certbot --version
certbot 1.23.0.dev0
$ sudo certbot certonly --register-unsafely-without-email --standalone -d bp.hyddns.xyz
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Requesting a certificate for bp.hyddns.xyz

Certbot failed to authenticate some domains (authenticator: standalone). The Certificate Authority reported these problems:
  Domain: bp.hyddns.xyz
  Type:   connection
  Detail: Fetching http://bp.hyddns.xyz/.well-known/acme-challenge/C8q1Z39O2Sfi7-4ZvrQsfBV4U9IXJY_tserQgI8y210: Timeout during connect (likely firewall problem)

Hint: The Certificate Authority failed to download the challenge files from the temporary standalone webserver started by Certbot on port 80. Ensure that the listed domains point to this machine and that it can accept inbound connections from the internet.

Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.

Considering that boringproxy also relies on HTTPS cert issued by Lets Encrypt. So I think that's probably why I failed.

[anderspitman]

Ok so it looks like boringproxy isn't the problem. Good for me; but doesn't help you.

This is very interesting. It seems that your ISP might be blocking LetsEncrypt entirely. Do you think that's possible?

[hongyi-zhao]

Another possible validation scheme is testing with other Let's Encrypt alike solutions, for example, the ones listed here:

Free Let's Encrypt Alternatives

    ZeroSSL. Freemium • Proprietary. Windows. Linux. ...
    SSL For Free. Free • Open Source. Online. SSL. ...
    AboutSSL. Free • Proprietary. Online. SSL. ...
    acme.sh. Free • Open Source. Mac. ...
    Buypass SSL. Free • Proprietary. Online. ...
    MEO File Encryption Software. Free • Proprietary. Mac. ...
    Parsslio. Free • Proprietary. Online.

But I must point out another thing that I haven't mentioned yet: The DNSPod, which is the DNS resolution service provider used by me, also can provide the SSL certificate for free, as shown below:

So, I wonder if I can let boringproxy work with the SSL certificate generated by DNSPod instead of creating and using the one supplied by Let's Encrypt at the setup stage.

[anderspitman]

Not sure. If DNSPod implements the ACME protocol, it should be possible to use it. boringproxy isn't currently configured to use CAs other than LetsEncrypt, but I'm pretty sure the underlying certmagic library supports it.

[hongyi-zhao]

This is very interesting. It seems that your ISP might be blocking LetsEncrypt entirely. Do you think that's possible?

Yep. I confirmed your conjecture: My ISP blocks some or all the well-known_ports on my VPS. Although the previous test of port 80/443 on VPS itself has no problem, accessing port 80/443 of VPS through the Internet from other places will fail.

Hence, another possible solution is to let boringproxy use other ports, such as 8080/4430, to set the letsencrypt certificate. Can boringproxy work this way? This actually becomes a feature support problem of letsencrypt itself, i.e., the question discussed here: Can you use LetsEncrypt with ports other than 80 and 443? I’ve been going through all the discussions on the above site, and it seems that LetsEncrypt doesn’t support this feature.

See here for the relevant discussion.

[anderspitman]

LetsEncrypt will not, to my knowledge, work with ports other than 80/443. Are you using a cloud VPS host or running it behind a residential ISP? This seems like a very strange limitation for a cloud VPS.

[hongyi-zhao]

Perhaps I should have told you more about the VPS used by myself, which is a self-built VPS running on my home router, using the dynamic public IP address assigned by ISP's FTTH 500Mbps line, combined with the ddns service provided by dnspod. This way, I obtained an almost free self built VPS, of course, DDNS service charges a certain low fee. So far, the only inconvenience I have encountered is the problem discussed here: Some well-known ports are blocked by ISP and cannot be used.

For more information, see the following screenshot of the low-cost and high-bandwidth private self-built VPS environment based on Proxmox Virtual Environment:

[anderspitman]

Yeah, so it sounds like boringproxy won't work for your configuration :frowning_face:. As a matter of fact, boringproxy was designed to work around exactly these sorts of issues, ie ISP firewalls, NAT, etc. But it requires that you have a server with unrestricted port 80/443 to run the server on, then you run the client on your home machines so they can tunnel out. Unfortunately a public VPS will cost money though.

[hongyi-zhao]

Now, with the help of danb35 and others, I've successfully deployed the required DNS TXT record and received the certificate manually as shown below:

$ sudo certbot --text --agree-tos --email you@example.com -d bp.hyddns.xyz --manual --preferred-challenges dns --expand --renew-by-default  --manual-public-ip-logging-ok certonly
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Requesting a certificate for bp.hyddns.xyz

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Please deploy a DNS TXT record under the name:

_acme-challenge.bp.hyddns.xyz.

with the following value:

2K29HzPOBJCeIPXhdbAo1BnnQ1oJxT5CeLFv7EoNOVE

Before continuing, verify the TXT record has been deployed. Depending on the DNS
provider, this may take some time, from a few seconds to multiple minutes. You can
check if it has finished deploying with aid of online tools, such as the Google
Admin Toolbox: https://toolbox.googleapps.com/apps/dig/#TXT/_acme-challenge.bp.hyddns.xyz.
Look for one or more bolded line(s) below the line ';ANSWER'. It should show the
value(s) you've just added.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Press Enter to Continue

Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/bp.hyddns.xyz/fullchain.pem
Key is saved at:         /etc/letsencrypt/live/bp.hyddns.xyz/privkey.pem
This certificate expires on 2022-03-20.
These files will be updated when the certificate renews.

NEXT STEPS:
- This certificate will not be renewed automatically. Autorenewal of --manual certificates requires the use of an authentication hook script (--manual-auth-hook) but one was not provided. To renew this certificate, repeat this same certbot command before the certificate's expiry date.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
If you like Certbot, please consider supporting our work by:
 * Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
 * Donating to EFF:                    https://eff.org/donate-le
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Before pressing the Enter key in the above step, first deploy the corresponding DNS TXT record and verify it as follows:

$ dig -t txt _acme-challenge.bp.hyddns.xyz +short
"2K29HzPOBJCeIPXhdbAo1BnnQ1oJxT5CeLFv7EoNOVE"

In addition, danb35 also told me the wonderful ACME client protocol automation tool, acme.sh. But I want to know: Can I let boringproxy use the certificate received above to further complete its related installation and configuration?

Regards, HZ

[anderspitman]

Good news! At first I assumed browser wouldn't support HTTPS on a different port from the one you used to get the cert, but then I got to thinking about it and wondered if it might actually work. Plus it's already been requested in #105 to support binding different ports. So I've added the arguments -http-port and -https-port. It required a few other changes as well, but I just tested using 8080/4443 and it worked great. Tested in Firefox. @hongyi-zhao can you try building from this branch and see if it works for you?

https://github.com/boringproxy/boringproxy/tree/allow-other-ports

Heads up the UI is significantly changed in the latest version.

[anderspitman]

Keep in mind that the certs will have to be in the location and format expect by certmagic. That might actually be quite tricky to do by hand and you'll have to figure it out. What might be best would be for boringproxy to take advantage of certmagic's ability to communicate with DNS providers directly. But I'm not really sure that's within scope of the project. I've opened #116 to track this.

[anderspitman]

@hongyi-zhao is your DNS provider in this list?

https://github.com/orgs/libdns/repositories?type=all

[hongyi-zhao]

@hongyi-zhao is your DNS provider in this list?

https://github.com/orgs/libdns/repositories?type=all

Yes. I use dnspod.

[hongyi-zhao]

can you try building from this branch and see if it works for you?

https://github.com/boringproxy/boringproxy/tree/allow-other-ports

$ git pull
$ git checkout allow-other-ports
$ git log -1
commit 30358d78087bfeaa60177d86faaa72092686944c (HEAD -> allow-other-ports, origin/allow-other-ports)
Author: Anders Pitman <tapitman11@gmail.com>
Date:   Mon Dec 20 12:56:50 2021 -0700

    Implement binding non-default ports

    Can now bind to ports other than 80/443, using the -http-port and
    -https-port arguments.

    Assuming you already have the certs you need, HTTPS even works.

    Unfortunately you can't get the certs automatically because
    LetsEncrypt doesn't support ports other than 80/443 as far as I
    know.

$ cd cmd/boringproxy/
$ go build
go: downloading github.com/mdp/qrterminal/v3 v3.0.0
../../ui_handler.go:19:12: pattern logo.png: no matching files found

[anderspitman]

Oh right. Install inkscape and run scripts/generate_logo.sh

[anderspitman]

You could also just put any PNG file at logo.png.

[hongyi-zhao]

$ bash scripts/generate_logo.sh 

** (inkscape:4009090): WARNING **: 06:44:40.794: Invalid option -o

For the inkscape version on my machine, the command should be as follows:

$ inkscape --version
Inkscape 0.92.5 (2060ec1f9f, 2020-04-08)

$ inkscape --help|grep png
  -e, --export-png=FILENAME                  Export document to a PNG file

$ inkscape -w 192 -h 192 logo.svg -e logo.png
Background RRGGBBAA: ffffff00
Area 0:0:100:100 exported to 192 x 192 pixels (184.32 dpi)
Bitmap saved as: logo.png

But this version still fails:

$ sudo ./boringproxy server -http-port 8000 -https-port 4430 -admin-domain bp.hyddns.xyz
2021/12/21 06:56:29 Starting up

Your sites will be served over HTTPS automatically using an automated CA.
By continuing, you agree to the CA's terms of service.
Please enter your email address to signify agreement and to be notified
in case of issues. You can leave it blank, but we don't recommend it.
  Email address: 
2021/12/21 06:57:29 bp.hyddns.xyz: obtaining certificate: [bp.hyddns.xyz] Obtain: [bp.hyddns.xyz] solving challenges: bp.hyddns.xyz: no solvers available for remaining challenges (configured=[http-01 tls-alpn-01] offered=[http-01 dns-01 tls-alpn-01] remaining=[dns-01]) (order=https://acme-v02.api.letsencrypt.org/acme/order/329670050/48872942840) (ca=https://acme-v02.api.letsencrypt.org/directory)

[anderspitman]

Yeah, as I said you'd have to get the certs manually (with certbot) then put them in the location/format required by certmagic. It's probably not worth going through that trouble. I may be able to put together something for doing DNS challenges. Might not be right away though. I'll let you know.

[anderspitman]

Ok @hongyi-zhao. I think that this may eventually be something that we'll have a better solution for, but I don't think it will be for a while. In the mean time, I hacked together a quick program that might solve your problem:

https://github.com/anderspitman/certgrabber

certgrabber can use the DNS challenge to get certificates and store them in the proper format in ~/.local/share/certmagic. Then boringproxy should be able to see them.

You can run it like this:

./certgrabber -domain bp.hyddns.xyz -provider dnspod -token <dnspod API token> -challenge-type dns-01

You'll need an API token from dnspod. Let me know how it goes!

[hongyi-zhao]

See the following:

$ certgrabber -domain bp.hyddns.xyz -provider dnspod -token <dnspod API token> -challenge-type dns-01
Getting cert for domain: bp.hyddns.xyz
bp.hyddns.xyz: obtaining certificate: [bp.hyddns.xyz] Obtain: [bp.hyddns.xyz] solving challenges: presenting for challenge: adding temporary record for zone xyz.: Create record err.Zone:xyz., Name: _acme-challenge.bp.hyddns, Value: fVbv_Lt3zu-ghHGastDOi-GbMNRVsXTX65Du2h_6eIo, Error:could not get domains: The login token ID is invalid, { TXT _acme-challenge.bp.hyddns fVbv_Lt3zu-ghHGastDOi-GbMNRVsXTX65Du2h_6eIo 0s 0} (order=https://acme-v02.api.letsencrypt.org/acme/order/324767840/48952190810) (ca=https://acme-v02.api.letsencrypt.org/directory)

[anderspitman]

Looks like there's something wrong with your dnspod token. I've never used dnspod sure not sure what it might be. It mentions "login token". Are there different types of tokens maybe?

[hongyi-zhao]

According to the description here, the correct DNSPod token format is as follows:

需要用这两个字段来组合成一个完整的 Token，组合方式为："ID,Token"
（用英文半角逗号分割），比如 ID 为：13490,ToKen为：6b5976c68aba5b14a0558b77c17c3932。
即完整的 Token 为：13490,6b5976c68aba5b14a0558b77c17c3932 。

So I tried as follows:

$ login_token="ID,Token"
$ certgrabber -domain bp.hyddns.xyz -provider dnspod -token $login_token -challenge-type dns-01
Getting cert for domain: bp.hyddns.xyz
bp.hyddns.xyz: obtaining certificate: [bp.hyddns.xyz] Obtain: [bp.hyddns.xyz] solving challenges: presenting for challenge: could not determine zone for domain "_acme-challenge.bp.hyddns.xyz": could not find the start of authority for _acme-challenge.bp.hyddns.xyz.: NOERROR (order=https://acme-v02.api.letsencrypt.org/acme/order/324767840/48975851400) (ca=https://acme-v02.api.letsencrypt.org/directory)

$ dig +short SOA hyddns.xyz
$ 
$ dig +short SOA bp.hyddns.xyz
$

Based on the above information, what should I do further/next?

[anderspitman]

Sorry I missed the notification for your last message. It looks like your DNS isn't properly configured. I have no idea how you would fix this, other than contacting dnspod and asking why there doesn't appear to be any SOA for the domain.

[hongyi-zhao]

I don't quite understand the SOA item. Please see the query results below:

$ dig hyddns.xyz +nssearch
SOA donald.dnspod.net. freednsadmin.dnspod.com. 1640235834 3600 180 1209600 180 from server 129.211.176.224 in 24 ms.
SOA donald.dnspod.net. freednsadmin.dnspod.com. 1640235834 3600 180 1209600 180 from server 129.211.176.187 in 24 ms.
SOA donald.dnspod.net. freednsadmin.dnspod.com. 1640235834 3600 180 1209600 180 from server 101.226.220.16 in 28 ms.
SOA donald.dnspod.net. freednsadmin.dnspod.com. 1640235834 3600 180 1209600 180 from server 61.151.180.44 in 28 ms.
SOA donald.dnspod.net. freednsadmin.dnspod.com. 1640235834 3600 180 1209600 180 from server 2402:4e00:1430:1102:0:9136:2b30:e554 in 28 ms.
SOA donald.dnspod.net. freednsadmin.dnspod.com. 1640235834 3600 180 1209600 180 from server 58.247.212.36 in 32 ms.
SOA donald.dnspod.net. freednsadmin.dnspod.com. 1640235834 3600 180 1209600 180 from server 58.247.212.48 in 32 ms.
SOA donald.dnspod.net. freednsadmin.dnspod.com. 1640235834 3600 180 1209600 180 from server 183.192.201.91 in 40 ms.
SOA donald.dnspod.net. freednsadmin.dnspod.com. 1640235834 3600 180 1209600 180 from server 2402:4e00:1020:1264:0:9136:29bc:87f9 in 40 ms.
SOA donald.dnspod.net. freednsadmin.dnspod.com. 1640235834 3600 180 1209600 180 from server 183.192.164.118 in 44 ms.
SOA donald.dnspod.net. freednsadmin.dnspod.com. 1640235834 3600 180 1209600 180 from server 162.14.24.230 in 188 ms.
SOA donald.dnspod.net. freednsadmin.dnspod.com. 1640235834 3600 180 1209600 180 from server 162.14.25.230 in 188 ms.

[anderspitman]

Honestly I don't know enough about dig to know what the issue is. Since certgrabber is just a very thin wrapper around certmagic, they might have a better idea what that error means.

[hongyi-zhao]

Since certgrabber is just a very thin wrapper around certmagic, they might have a better idea what that error means.

I've given a glimpse of its usage. It seems to be a go library rather than a command line tool, so I don't know how to test it - my Golang programming experience is very limited.

[anderspitman]

Not sure what you mean exactly. The error being displayed is directly from certmagic. If you show it to them and link back to this issue they might be able to give some clues about the underlying issue.

[hongyi-zhao]

Thank you for your clarification. I mean, I still can't figure out how to test with certmagic under command line instead of programmatically by written go code snippet.

[anderspitman]

You don't need to test certmagic. You just need to find out what that specific error is about, then we can decide where to go from there.

[hongyi-zhao]

Finally, I figured out that the culprit is the following option enabled in my dnsmasq configuration file (Public/anti-gfw/dns/dnsmasq/conf/dnsmasq.conf), see here for the relevant comment by me:

# Uncomment this to filter useless windows-originated DNS requests
# which can trigger dial-on-demand links needlessly.
# Note that (amongst other things) this blocks all SRV requests,
# so don't use it if you use eg Kerberos, SIP, XMMP or Google-talk.
# This option only affects forwarding, SRV records originating for
# dnsmasq (via srv-host= lines) are not suppressed by it.
filterwin2k

Commenting out the above option solved the problem:

$ dig +short SOA bp.hyddns.xyz 
www.hyddns.xyz.
$ dig +short SOA hyddns.xyz 
donald.dnspod.net. freednsadmin.dnspod.com. 1640510300 3600 180 1209600 180

$ login_token="ID,Token"
$ certgrabber -domain bp.hyddns.xyz -provider dnspod -token $login_token -challenge-type dns-01
Getting cert for domain: bp.hyddns.xyz
$ 

Regards, HZ

[anderspitman]

Nice! Any outlying issues?

[hongyi-zhao]

Any outlying issues?

What should I do next? Sadly to say, currently, the vps has been blocked by ISP, so I can’t do any further testing the VPS from the public network.

[anderspitman]

Now that you have the certs for certmagic, you should be able to run boringproxy server with custom ports, and hopefully it will work. You'll need to build the current master. I just pushed some changes a few hours ago to fix running on different ports. See #123.

[hongyi-zhao]

I confirmed your above argumentation by the following steps:

$ login_token="ID,Token"
$ certgrabber -domain bp.hyddns.xyz -provider dnspod -token $login_token -challenge-type dns-01
Getting cert for domain: bp.hyddns.xyz
$ git log -1
commit 46bb670b8fd1afbcbf8bb203c6934e586db78fe5 (HEAD -> master, origin/master, origin/HEAD)
Author: Anders Pitman <tapitman11@gmail.com>
Date:   Sat Jan 1 16:32:51 2022 -0700

    Disable ACME when using custom ports

    LetsEncrypt currently only supports ports 80/443 for ACME, so if
    custom ports are used we need to disable automatically getting
    certs.

    Also changed it to only emit a warning when ports aren't
    reachable from the internet, rather than failing.

$ cd cmd/boringproxy
$ go build
$ ./boringproxy server -http-port 8000 -https-port 4430 -admin-domain bp.hyddns.xyz
2022/01/02 11:20:04 Starting up
WARNING: LetsEncrypt only supports HTTP/HTTPS ports 80/443. You are using 8000/4430. Disabling automatic certificate management
2022/01/02 11:20:05 Ready

But as you can see, this method will disable LetsEncrypt automatic certificate management, which is somewhat inconvenient in the long run.

[anderspitman]

There's nothing you can do about that at the moment. boringproxy only understands the HTTP-01 and TLS-ALPN-01 ACME challenges, which won't work for you because your ISP is blocking 80/443. So you have to use something like certgrabber to use the DNS-01 challenge.

It's less convenient but probably your only option. Is it at least working?

[hongyi-zhao]

Is it at least working?

I'll report after I try.

[andrewdhastings]

I'm having the same issue, however, I'm using a raw DigitalOcean Ubuntu 20 server. DNS is pointing to the server from Namecheap. I can get a cert just fine with Caddy2, but for some reason when I try to run boringproxy it always errors out with this error:

obtaining certificate: [themodern.cloud] Obtain: [themodern.cloud] solving challenges: themodern.cloud: no solvers available for remaining challenges (configured=[http-01 tls-alpn-01] offered=[http-01 dns-01 tls-alpn-01] remaining=[dns-01]) (order=https://acme-v02.api.letsencrypt.org/acme/order/524700477/88384052786) (ca=https://acme-v02.api.letsencrypt.org/directory)

Any Ideas on what else would be causing the error?

[anderspitman]

@andrewdhastings pretty sure this is usually due to a firewall somewhere blocking port 443. Does boringproxy server print any warnings when it starts up? Have you verified that you can run another webserver on port 443 and have it accessed as expected?

[andrewdhastings]

Sorry, looks like I'm just an idiot. I thought I was using bare ubuntu, but actually was using a docker setup. (normally my go-to)

Anyways, I just used the docker-compose options and everything is up and running.

[natecovington]

I'm having a similar issue: no solvers available for remaining challenges (configured=[tls-alpn-01] offered=[http-01 dns-01 tls-alpn-01] remaining=[http-01 dns-01]) This occurred after I had a power outage at home, after trying to reboot my cloud VPS and home machines to bring things up again, I wound up having to remove manually a few tunnels and recreate them to get everything working again. I have around 20 tunnels configured right now and it seems like I'm hitting some sort of limit with Let's Encrypt.

When I look up my domain here: https://crt.sh/

I can see that boring proxy sometimes get stuck and generates ~15 certificates in a day, I'm guessing there is some magic sequence that I'm supposed to do in order to bring the system back up in the correct order? Reboot the VPS first, then the homeservers?

I'm also not sure if one of these crontabs will help me, where it checks and restarts the boring proxy service: https://forum.indiebits.io/t/connection-refused-have-to-restart-service-periodically/239/6 https://github.com/boringproxy/boringproxy/issues/41

Another question, does it matter if I run Boring Proxy inside a docker container vs. native? If I'm running docker containers on my homserver does that mean I need to run it inside a docker container on my VPS?

[anderspitman]

I'm pretty sure you shouldn't be requesting that many certs in a day. I'm guessing that you're not saving them between boringproxy server startups. Maybe you're running in an ephemeral docker container that keeps getting recreated from scratch? Other than that, docker is just a containerization mechanism. boringproxy should happily run in or out of docker. But the way you use docker can effect it. I definitely recommend getting as deep an understanding of docker as possible. I've found it to be very worthwhile knowledge.

[natecovington]

I'm not using BoringProxy in a docker container, it's running directly in /systemd/system

Am I supposed to be doing something to manually save the certificates between startups? It seems like they're saved in the JSON - db file?

And to clarify, it wasn't trying to generate new certificates for all my apps, I can see on crt.sh when I search for my subdomains, it was trying to create the same SSL certificate for the same subdomain repeatedly, maybe 10 minutes apart. One subdomain I'm looking at in particular, in just over 2 months it generated 19 certificates. (see attached)

Edit: I'm looking at my screenshot and April 7 is the day I set up my Boring Proxy VPS. Before that I was using a different reverse proxy setup. So maybe I was running into issues because of that migration?

[natecovington]

I was able to generate some better logs of what's happening here, the no solvers available for remaining challenges error is actually one of the last errors that I'm getting (below). I've got a lot of tunnels set up so it's a bit long, but I'm not requesting it to create all these certificates at once, it's getting stuck somehow and looping?

First it creates and bores the tunnels, then it starts getting the failed to dial and handshake messages, then the part I'm seeing that's interesting is the [INFO][FileStorage:/home/redacted/.local/share/certmagic] Lock for 'issue_cert_matrix.example2.com' is stale piece.

When I go into that .local/share/certmagic folder there are three of my certs listed in there right now, maybe they're the ones it'll have trouble with the next time it runs?

/home/(redacted)/boring/boringproxy-linux-x86_64 client -server boring.example.net -token (redacted) -client-name (redacted) -user (redacted)
2023/06/12 21:44:51 SyncTunnels
2023/06/12 21:44:51 New tunnel jvb.boring.example.net
2023/06/12 21:44:51 New tunnel serpbear.boring.example.net
2023/06/12 21:44:51 New tunnel status.example.net
2023/06/12 21:44:51 New tunnel synapse-admin.boring.example.net
2023/06/12 21:44:51 New tunnel example.net
2023/06/12 21:44:51 New tunnel element.lppamonroe.com
2023/06/12 21:44:51 BoreTunnel jvb.boring.example.net
2023/06/12 21:44:51 BoreTunnel element.lppamonroe.com
2023/06/12 21:44:51 BoreTunnel serpbear.boring.example.net
2023/06/12 21:44:51 New tunnel matrix.lppamonroe.com
2023/06/12 21:44:51 New tunnel neptune.boring.example.net
2023/06/12 21:44:51 New tunnel www.example.net
2023/06/12 21:44:51 BoreTunnel matrix.lppamonroe.com
2023/06/12 21:44:51 BoreTunnel www.example.net
2023/06/12 21:44:51 New tunnel matrix.example2.com
2023/06/12 21:44:51 New tunnel change.boring.example.net
2023/06/12 21:44:51 New tunnel cloud.boring.example.net
2023/06/12 21:44:51 New tunnel cloud.example2.com
2023/06/12 21:44:51 New tunnel element.example.net
2023/06/12 21:44:51 New tunnel element.example2.com
2023/06/12 21:44:51 New tunnel matrix.example.net
2023/06/12 21:44:51 BoreTunnel neptune.boring.example.net
2023/06/12 21:44:51 New tunnel jitsi.boring.example.net
2023/06/12 21:44:51 BoreTunnel jitsi.boring.example.net
2023/06/12 21:44:51 BoreTunnel status.example.net
2023/06/12 21:44:51 BoreTunnel example.net
2023/06/12 21:44:51 BoreTunnel matrix.example2.com
2023/06/12 21:44:51 BoreTunnel synapse-admin.boring.example.net
2023/06/12 21:44:51 BoreTunnel change.boring.example.net
2023/06/12 21:44:51 BoreTunnel element.example.net
2023/06/12 21:44:51 BoreTunnel cloud.boring.example.net
2023/06/12 21:44:51 BoreTunnel element.example2.com
2023/06/12 21:44:51 BoreTunnel cloud.example2.com
2023/06/12 21:44:51 BoreTunnel matrix.example.net
2023/06/12 21:44:51 BoreTunnel error:  Failed to dial: ssh: handshake failed: read tcp 192.168.1.50:58088->1.2.3.4:22: read: connection reset by peer
2023/06/12 21:44:51 BoreTunnel error:  Failed to dial: ssh: handshake failed: read tcp 192.168.1.50:58244->1.2.3.4:22: read: connection reset by peer
2023/06/12 21:44:51 BoreTunnel error:  Failed to dial: ssh: handshake failed: read tcp 192.168.1.50:58186->1.2.3.4:22: read: connection reset by peer
2023/06/12 21:44:51 BoreTunnel error:  Failed to dial: ssh: handshake failed: read tcp 192.168.1.50:58142->1.2.3.4:22: read: connection reset by peer
2023/06/12 21:44:53 BoreTunnel error:  Unable to register tcp forward for 0.0.0.0:3014 ssh: tcpip-forward request denied by peer
2023/06/12 21:44:53 BoreTunnel error:  Unable to register tcp forward for 0.0.0.0:3011 ssh: tcpip-forward request denied by peer
2023/06/12 21:44:53 [INFO][FileStorage:/home/redacted/.local/share/certmagic] Lock for 'issue_cert_matrix.example2.com' is stale (created: 2023-06-12 15:39:43.673943723 -0400 EDT, last update: 2023-06-12 19:55:06.258987157 -0400 EDT); removing then retrying: /home/redacted/.local/share/certmagic/locks/issue_cert_matrix.example2.com.lock
2023/06/12 21:44:53 [INFO][FileStorage:/home/redacted/.local/share/certmagic] Lock for 'issue_cert_element.example2.com' is stale (created: 2023-06-12 15:39:43.674158123 -0400 EDT, last update: 2023-06-12 19:55:06.258986997 -0400 EDT); removing then retrying: /home/redacted/.local/share/certmagic/locks/issue_cert_element.example2.com.lock
2023/06/12 21:44:55 CertMagic error at startup
2023/06/12 21:44:55 cloud.boring.example.net: obtaining certificate: [cloud.boring.example.net] Obtain: [cloud.boring.example.net] solving challenges: cloud.boring.example.net: no solvers available for remaining challenges (configured=[tls-alpn-01] offered=[http-01 dns-01 tls-alpn-01] remaining=[http-01 dns-01]) (order=https://acme-v02.api.letsencrypt.org/acme/order/redacted) (ca=https://acme-v02.api.letsencrypt.org/directory)
2023/06/12 21:44:55 CertMagic error at startup
2023/06/12 21:44:55 synapse-admin.boring.example.net: obtaining certificate: [synapse-admin.boring.example.net] Obtain: [synapse-admin.boring.example.net] solving challenges: synapse-admin.boring.example.net: no solvers available for remaining challenges (configured=[tls-alpn-01] offered=[http-01 dns-01 tls-alpn-01] remaining=[http-01 dns-01]) (order=https://acme-v02.api.letsencrypt.org/acme/order/redacted7) (ca=https://acme-v02.api.letsencrypt.org/directory)
2023/06/12 21:44:55 CertMagic error at startup
2023/06/12 21:44:55 jvb.boring.example.net: obtaining certificate: [jvb.boring.example.net] Obtain: [jvb.boring.example.net] solving challenges: jvb.boring.example.net: no solvers available for remaining challenges (configured=[tls-alpn-01] offered=[http-01 dns-01 tls-alpn-01] remaining=[http-01 dns-01]) (order=https://acme-v02.api.letsencrypt.org/acme/order/redacted) (ca=https://acme-v02.api.letsencrypt.org/directory)
2023/06/12 21:44:55 CertMagic error at startup
2023/06/12 21:44:55 matrix.lppamonroe.com: obtaining certificate: [matrix.lppamonroe.com] Obtain: [matrix.lppamonroe.com] solving challenges: matrix.lppamonroe.com: no solvers available for remaining challenges (configured=[tls-alpn-01] offered=[http-01 dns-01 tls-alpn-01] remaining=[http-01 dns-01]) (order=https://acme-v02.api.letsencrypt.org/acme/order/redacted) (ca=https://acme-v02.api.letsencrypt.org/directory)
2023/06/12 21:44:55 CertMagic error at startup
2023/06/12 21:44:55 element.example.net: obtaining certificate: [element.example.net] Obtain: [element.example.net] solving challenges: element.example.net: no solvers available for remaining challenges (configured=[tls-alpn-01] offered=[http-01 dns-01 tls-alpn-01] remaining=[http-01 dns-01]) (order=https://acme-v02.api.letsencrypt.org/acme/order/redacted) (ca=https://acme-v02.api.letsencrypt.org/directory)
2023/06/12 21:44:55 CertMagic error at startup
2023/06/12 21:44:55 element.example2.com: renewing certificate: [element.example2.com] Renew: [element.example2.com] solving challenges: element.example2.com: no solvers available for remaining challenges (configured=[tls-alpn-01] offered=[http-01 dns-01 tls-alpn-01] remaining=[http-01 dns-01]) (order=https://acme-v02.api.letsencrypt.org/acme/order/redacted) (ca=https://acme-v02.api.letsencrypt.org/directory)
2023/06/12 21:44:55 CertMagic error at startup
2023/06/12 21:44:55 matrix.example2.com: renewing certificate: [matrix.example2.com] Renew: [matrix.example2.com] solving challenges: matrix.example2.com: no solvers available for remaining challenges (configured=[tls-alpn-01] offered=[http-01 dns-01 tls-alpn-01] remaining=[http-01 dns-01]) (order=https://acme-v02.api.letsencrypt.org/acme/order/redacted) (ca=https://acme-v02.api.letsencrypt.org/directory)
2023/06/12 21:44:56 CertMagic error at startup
2023/06/12 21:44:56 example.net: obtaining certificate: [example.net] Obtain: [example.net] solving challenges: example.net: no solvers available for remaining challenges (configured=[tls-alpn-01] offered=[http-01 dns-01 tls-alpn-01] remaining=[http-01 dns-01]) (order=https://acme-v02.api.letsencrypt.org/acme/order/redacted) (ca=https://acme-v02.api.letsencrypt.org/directory)
2023/06/12 21:44:56 CertMagic error at startup
2023/06/12 21:44:56 element.lppamonroe.com: obtaining certificate: [element.lppamonroe.com] Obtain: [element.lppamonroe.com] solving challenges: element.lppamonroe.com: no solvers available for remaining challenges (configured=[tls-alpn-01] offered=[http-01 dns-01 tls-alpn-01] remaining=[http-01 dns-01]) (order=https://acme-v02.api.letsencrypt.org/acme/order/redacted) (ca=https://acme-v02.api.letsencrypt.org/directory)
2023/06/12 21:44:56 CertMagic error at startup
2023/06/12 21:44:56 neptune.boring.example.net: obtaining certificate: [neptune.boring.example.net] Obtain: [neptune.boring.example.net] solving challenges: neptune.boring.example.net: no solvers available for remaining challenges (configured=[tls-alpn-01] offered=[http-01 dns-01 tls-alpn-01] remaining=[http-01 dns-01]) (order=https://acme-v02.api.letsencrypt.org/acme/order/redacted) (ca=https://acme-v02.api.letsencrypt.org/directory)