Support for FQDNs under .onion

cpu commented 4 years ago

Background

Historically the CABF baseline requirements have considered FQDNs with .onion as the rightmost label "internal names" because the TLD is not registered in IANA's root zone database:

Internal Name: A string of characters (not an IP address) in a Common Name or Subject Alternative Name field of a Certificate that cannot be verified as globally unique within the public DNS at the time of certificate issuance because it does not end with a Top Level Domain registered in IANA’s Root Zone Database.

To support HTTPS for .onion sites the "Extended Validation Certificate Guidelines" specify a carve-out that allows issuing EV certificates for .onion FQDNs if extra steps specified in an Appendix are followed:

For a Certificate issued to a Domain Name with .onion in the right-most label of the Domain Name, the CA SHALL confirm that, as of the date the Certificate was issued, the Applicant’s control over the .onion Domain Name in accordance with Appendix F.

The EV requirement and the additional cabf-TorServiceDescriptor certificate extension in Appendix F were largely intended to address weak cryptography used by tor v2 .onion addresses. Since the v3 address specification addresses those weaknesses there is now a proposal in the CABF validation working group in the works by Wayne Thayer of Mozilla to change the CABF baseline requirements to allow DV issuance for .onion FQDNs that are implemented with the "v3" .onion address specification.

Implementation notes

If a ballot based on this proposal were to be approved Let's Encrypt/Boulder could in theory issue for .onion domains. This issue is to track some of the things we'd need to figure out to support that. It isn't a commitment to implement, but a collection of notes particularly focused on some challenges we would face.

Domain control

From an operational perspective adding a tor daemon adjacent to each of our VA instances to allow outbound tor communication to .onion sites to verify an agreed upon change to a website (via HTTP-01) is a substantial burden. It's a complex daemon in a memory unsafe language and we would need new monitoring and a lot of general SRE work to deploy it.

We would probably prefer to implement proof of domain control support based on the "CSR validation" method described in Wayne's draft ballot since there would be no requirement to make an outbound connection to the .onion FQDN. Probably implementing the CSR validation properly would require the definition of a new ACME challenge type since there is a need to relay and verify additional information in the CSR (the caSigningNonce and the applicantSigningNonce).

Rate limits

Our rate limits are primarily based around the notion that folks can't create infinite eTLD+1's because of the cost associated with domain registration. In contrast .onion names can be minted infinitely without any substantial cost. We would need to think of a way to apply fair rate limits for .onion names in order to ensure fair distribution of our resources.

We could have a global rate limit for .onion names but that invites DoSing the availability of these certificates for others by continually issuing until the global rate limit is reached. Making the rate limit per-account would shift the burden to the new registration per IP rate limit which today is quite generous. Similarly we could affect a rate limit by requester IP but this would also be generous (and IPv6 addresses are cheap/plentiful).

Gauging demand

It's difficult to gauge subscriber demand for .onion certificates because to date the requirement for EV certificates with a custom extension meant only ~1 commercial CA implemented support and the cost was likely prohibitive for most users. Without knowing the demand it's difficult to make a decision on how important allocating resources to the work required is.

rolandshoemaker commented 4 years ago

Developing a ACME challenge for the CSR verification method seems complicated mainly because of the Verified Method of Communication requirement. I'm not sure there is an easy way we could automate the verification as required in the EVGLs (section 11.5.1 and 11.5.2).

It seems like the proposed ballot leaves out the Verified Method of Communication definition from the BRs (which has a similar, but notably different, Reliable Method of Communication), I think it'd need to be updated to actually account for this which could open the door for an easier ACME challenge.

jsha commented 4 years ago

Also worth noting: In term of translation into ACME, it probably makes sense for the authorizations to be for a different identifier type (e.g. onion) since it's not verified using the usual DNS. The order would then get turned into a certificate with the .onion name in a dNSName entry in subjectAltName, but keeping the types separated until final issuance would make it a little easier to do type switches in the validation code.

cpu commented 4 years ago

Also worth noting: In term of translation into ACME, it probably makes sense for the authorizations to be for a different identifier type (e.g. onion) since it's not verified using the usual DNS.

Good point, updated the issue title accordingly.

orangepizza commented 4 years ago

Because ACME protocol server only ask CSR after domain client asked passed validation, Is it possible to do CSR based challenge in current protocol? otherwise VA have to be able to talk to the client to get the client. Actually, we may want two CSR from client, one signed with onion public key for challenge(as the ballet's challenge requires that), and for finalized the challenge (so end user can use any key that isn't onion key) but it's hard to avoid VA's direct talk with acme client it we don't want to run Tor daemon on VA.

xiaohuilam commented 4 years ago

Because ACME protocol server only ask CSR after domain client asked passed validation, Is it possible to do CSR based challenge in current protocol? otherwise VA have to be able to talk to the client to get the client. Actually, we may want two CSR from client, one signed with onion public key for challenge(as the ballet's challenge requires that), and for finalized the challenge (so end user can use any key that isn't onion key) but it's hard to avoid VA's direct talk with acme client it we don't want to run Tor daemon on VA.

Seems nope, and it's a related question of https://community.letsencrypt.org/t/why-acme-requires-domain-auth-first-before-csr/98482

cpu commented 4 years ago

Since the v3 address specification addresses those weaknesses there is now a proposal in the CABF validation working group in the works by Wayne Thayer of Mozilla to change the CABF baseline requirements to allow DV issuance for .onion FQDNs that are implemented with the "v3" .onion address specification.

Quick update: The CABF ballot based on this proposal has passed.

Repentinus commented 4 years ago

Our rate limits are primarily based around the notion that folks can't create infinite eTLD+1's because of the cost associated with domain registration. In contrast .onion names can be minted infinitely without any substantial cost. We would need to think of a way to apply fair rate limits for .onion names in order to ensure fair distribution of our resources.

One way to solve this issue would be to require at least one non-onion Subject Alternative Name or Common Name in any certificate issued. The usual limits could then be applied to these names and onion names ignored for rate-limiting purposes.

Granted, this would not cater to people who do not want to associate their hidden services with a clearnet identity, but this demographic's only benefit from CA-issued certificates would be to have browsers enable secure content restrictions. Given that users of such services generally do not have anything other than an onion v3 address to identify them, their users should have no reason not to trust a self-signed certificate, optionally signed with the onion service's Tor private key.

If widely adopted, this would also permit people to write browser extensions (and other software) that display the subject CN or the first SAN in addition to the onion service address in the UI of a particular application. Such software could further choose to automatically trust self-signed certificates for onion services that have been signed with the onion service's Tor private key.

fancsali commented 3 years ago

As someone looking at it from the user side, I reckon the feature would be greatly appreciated by many. Also, I reckon, most of us could live with a stricter rate limit for plain .onion certificates.

However, mandating that also a "normal" DNS name needs to be present is pretty much defeating the purpose of a hidden service.

Also, I myself have a couple of home automation project, that I do not want to expose to the open internet, hence there's no way, I can have a proper DNS name for it. The only way in for maintenance and emergency is a Tor hidden service. I reckon, that's another valid use-case for onion-only names.

maltfield commented 3 years ago

First, thanks for putting ticket together :)

I imagine we're on the same page, but because there's so much "you don't need https for .onion sites; it provides no security benefit" opinions expressed on the internet, I wanted to add our use-case for this: it makes sense to add an https cert for Onion Services when adding a .onion domain as an alias to existing, complex sites that have http -> https redirects buried everywhere

Personally, I'm trying to add a .onion as a secondary domain for all my existing wordpress sites. Years ago, I migrated from http to https when Let's Encrypt first came out. When I made that transition, I checked all the boxes to "redirect http to https" everywhere I could: in my web server's config, in the caching reverse proxies, in the CMS core config, in various plugins, themes, etc. Now that I'm trying to add a .onion to my existing websites, I'm finding that some of my sites work OK, but others stubbornly refuse to serve traffic over http. They just 301 redirect to https://xyz.onion (which of course doesn't work). Isolating and changing this behavior is non-trivial.

There's huge privacy benefits to be gained by site admins making their existing websites accessible to tor users through onion services, but--since all our great efforts to migrate from http to https in the past years--it's not always trivial to just point a .onion at a website and have all the infrastructure we hardened just accept serving over http again.

If ACME could support issuing certs for .onion sites, this would lower the barrier of entry for sysadmins to be able to bring their existing websites onto the tor network, which would be another huge benefit for the privacy of Internet users everywhere.

maltfield commented 3 years ago

Besides redirects as mentioned above, here's a few other technologies that modern browsers may not let clients use over http when a sysadmin tries to make their existing clearnet site accessible over a tor onion service:

cookie handling (Header always edit Set-Cookie (.*) "$1; HTTPOnly;Secure or set_cookie_flag HttpOnly secure;)
whether to permit access to camera and microphone devices (WebRTC)

In response to the original post's "gauging demand" section, the following resources may be helpful:

Mainstream websites using https on a `.onion` domain

@alecmuffett maintains a repo (automatically updated daily) of mainstream websites (>10,000 users) using .onion domains. Currently I count 132 websites. Here's a short sample:

At the time of writing, all of the above websites use DigiCert to secure their .onion domains with https. Only one of them (facebook) also use DigiCert for their clearnet domain as well.

The vast majority of these webistes do not use https for their .onion domains -- probably because of how expensive it is to obtain a distinct certificate for their .onion from DigiCert. Consequently, they will have issues using some technology (eg WebRTC) on their .onion domains until Let's Encrypt can solve this issue.

List of certificates issued for `.onion` domains

You can also query for certs issued for .onion domains using crt.sh:

https://crt.sh/?q=%25.onion

Motophan commented 3 years ago

My issue is that a lot of existing infrastructure needs to be modified to work with http only. Remember let's encrypt does not verify entities if they are good, bad, or neutral. Let's encrypt only adds the SSL bits and nothing else. That's all I want, I don't care about EV I just need something so a TLS connection can be made and the browser won't throw an error. Please just let me register a cert, digicert has too many requirements I just want the SSL bits I don't have a clearnet site.

fancsali commented 3 years ago

Also, I think, ZeroSSL started to issue 90-day certificates for .onion addresses. I reckon, this is the general direction in the industry now...

maltfield commented 3 years ago

@fancsali can you please post a link with more info about ZeroSSL's .onion cert support?

raucao commented 3 years ago

I just tried creating a cert for a .onion domain over at ZeroSSL and even though I was able to go through the process, the validation did not work in the end.

fancsali commented 3 years ago

@fancsali can you please post a link with more info about ZeroSSL's .onion cert support?

Well, you can type in a .onion address, and it'll happily let you continue with the process.

I just tried creating a cert for a .onion domain over at ZeroSSL and even though I was able to go through the process, the validation did not work in the end.

Interesting, I might have just assumed, they do, as they would let me go through the whole process. Didn't bother to set up an HTTP server last time, though. Nevertheless, now I did, and it seems to fail, as described.

My bad then.

orangepizza commented 3 years ago

They even try to verify certificate for internal ip (10.123.123.123) and just 192.168.0.0/16 get special treatment, I think their web is just too lex at inpit varification

fancsali commented 3 years ago

They even try to verify certificate for internal ip (10.123.123.123) and just 192.168.0.0/16 get special treatment, I think their web is just too lex at inpit varification

I am afraid so, apologies for assuming otherwise... ;)

I still see the value in certificates for the .onion special-use domains.

Motophan commented 3 years ago

Is it possible to just do the SSL portion w/o ev validation? Should I make a seperate issue for this?

raucao commented 3 years ago

Is it possible to just do the SSL portion w/o ev validation? Should I make a seperate issue for this?

That's exactly what this issue is for in the first place.

marmarek commented 3 years ago

However, mandating that also a "normal" DNS name needs to be present is pretty much defeating the purpose of a hidden service.

Thanks to the dns-01 challenge, that name doesn't need to point at your server. In fact, I think it doesn't need to have an A/AAAA record at all. That isn't a solution for those who need to host a website anonymously, but for many other use cases (like yours) it isn't an issue.

In fact I'd say, having a clearnet DNS name in the same cert is even desirable for the cases where onion service is used to protect the client location/identity, and not so much the server one. This way you can additionally validate if the onion address you use really corresponds to the clearnet one.