Unable to obtain LE Cert on internal subdomain

[bugs181]

Getting strange behavior which causes Tyger2 to fail to startup. This happened after switching the docker branch over to beta and adding a new application. Oddly enough, it's trying to use letsencrypt even though my DNS is hosted at cloudflare.

Sun Jun 14 08:16:00 2020 - [emperor] vassal /apps/Tyger2/install/uwsgi.ini is ready to accept requests
2020/06/14 08:16:01 [INFO] [sub.domain.com] acme: Obtaining bundled SAN certificate
2020/06/14 08:16:02 [INFO] [sub.domain.com] acme: Obtaining bundled SAN certificate
2020/06/14 08:16:03 [INFO] [sub.domain.com] acme: Obtaining bundled SAN certificate
2020/06/14 08:16:05 [INFO] [sub.domain.com] acme: Obtaining bundled SAN certificate
2020/06/14 08:16:06 [INFO] [sub.domain.com] acme: Obtaining bundled SAN certificate
2020/06/14 08:16:07 failed to obtain certificate: acme: error: 429 :: POST :: https://acme-v02.api.letsencrypt.org/acme/new-order :: urn:ietf:params:acme:error:rateLimited :: Error creating new order :: too many failed authorizations recently: see https://letsencrypt.org/docs/rate-limits/, url: 

Not sure why it's trying to Obtain the SAN certificate so many times. Think we can dig into this?

[morph1904]

Hi Bugs,

Caddy uses the letsencrypt authority to sign on all domains where SSL is on by default. When you add an address to Tyger, we add this address as a host in the caddyfile, caddy then creates an acme auth directory and asks letsencrypt to provide a cert for the domain, it does this by passing a key into the acme directory and asks letsencrypt to check there for it on port 80. If letsencrypt finds it and is happy it returns a signed cert which caddy stores and uses to provided https on that domain. If that check fails, you may encounter the lets encrypt rate limiter. This is usually caused by letsencrypt not being able to verify the domain because port 80 is not open or pointing to the caddy instance. If you look further back in the logs you may see an authorisation failing for another reason other than the rate limiter.

Currently Tyger does not support DNS verification (although the option is present in the UI to choose the DNS provider this is currently purely cosmetic) This is due to issues in compiling the various modules required to support this for the Caddy binary. The new version I am working on may support this when I get to it as Caddy 2 is the basis for the new version which has a different architecture.

[bugs181]

Thank you for the detailed write up on how the system works. You are right that their was a previous error although it's fairly unhelpful to me. It may be enough from your perspective being the engineer on it however to an end user, offers no insight.

The aforementioned error is:

2020/06/14 15:32:40 [INFO] Unable to deactivated authorizations: https://acme-v02.api.letsencrypt.org/acme/authz-v3/...

---

I'm thinking this likely has something to do with my particular DNS set up. I've recently started using my home brewed script, mentioned in another issue, which points the subdomains to LAN addresses. I was forwarding the primary domain correctly but it appears that even the subdomains must point to the WAN address also for the port 80 incoming check.

Too bad there's not an easy way to verify the server authenticity for SSL without exposing it to the public internet. If we pay for a particular domain, there should be a much easier solution to register a wildcard for any subdomains we would like. Perhaps this is also something Caddy 2 and Tyger2 could solve in the future.

[morph1904]

There is an easy way which is DNS authorisation. Basically instead of letsencrypt verifying the endpoint to see if you own it, if verifies that you have control over the domains zone file instead by adding a txt record to your provider and then reading that txt record through dns.

Unfortunately the established standard is that internal routing uses internal DNS servers in most environments which as you say may be the cause of your issue.

I understand the errors are not very friendly. These are generated by Caddy and letsencrypt so Tyger has little control over what they display. I am looking at improving that. I will have an alpha of the new version ready in a week or two that I want to have DNS verification working on this should help, though I am not sure if it will fully solve your issue, I would be grateful if you were able to assist with testing it, as your setup is an interesting configuration that would be good to get functioning with Tyger. I would also be interested in looking at integrating your script in some fashion too.

[bugs181]

I would be grateful if you were able to assist with testing it

Absolutely! I want Tyger to succeed. It has been by far the best solution I've used to date. And trust me when I say that I've tried them all. I've tried Caddy 1 and 2 barebones, nginx, a lot of obscure ones, and even rolled my own with limited success. Tyger has blown all of them out of the water in terms of ease of use and getting a setup up and running with very limited fuss.

As you said, I have a very interested setup. What makes Tyger an ideal fit for my needs is that I'm using this in a self hosted environment and whatever works eventually gets scaled out to my business needs in a corporate environment. I have the end say on what gets deployed to my company's servers and I'd LOVE for Tyger to be that fit for us. If successful, it's quite likely you'll most likely receive some funding from our end for future development and trouble shooting.

As for the script integration, that could work out quite well for a lot of users using Cloudflare for their hosting needs. Very minimal amount of coding required on both our parts to make it work for a lot of things. Consumer edge cases, adding scripting/job abilities to Tyger for letting users update their DNS suited just for their needs, etc. I've already built zone support and specific DNS endpoints into the script, so it would have a lot of flexibility for users.

[morph1904]

@bugs181 That sounds awesome, any feedback, suggestions, features, testers are all always welcome. As the issue you are experiencing is not technically a bug or problem with Tyger I will close this for now. I will be more than happy to keep communcations open though.