Escalating manual review, if need be

[konklone]

We just bought our first .gov certificate through SSLMate, but we got flagged for some kind of manual review, upstream:

I imagine it's because we're buying a .gov, in which case this will come up all the time. The cert in question is replacing one that expires in 10 days. If need be, is there any way to escalate the review, or to communicate an understanding upstream that our account is a .gov account (our account having a .gov email address might help).

[NoahKunin]

To echo above, it also might be helpful that we work not just at any Government agency, but at the agency (General Services Administration) that manages the .gov space itself.

[AGWA]

The manual review is "supposed" to be done within 24 hours, though I'm not sure if that includes weekends.

In the meantime, I've changed 18F's account to use RapidSSL instead of PositiveSSL. Would you mind resubmitting the order so we can see if RapidSSL is any more accepting of .gov certs? (I'll cancel whichever order ends up being slower.) If RapidSSL holds it for manual review too, we'll get the ball rolling with escalating this upstream and figuring out how to avoid this with future .gov purchases.

[konklone]

Well, I just had a chance to check, and PositiveSSL approved myra.treasury.gov, so I won't resubmit. But I guess we'll see the next time how RapidSSL manages it. Hopefully that next time will be very soon.

[konklone]

Thanks for the quick response and for helping us with our account!

[konklone]

And here's our first .gov certificate in production from SSLMate, at myra.treasury.gov:

Thanks again!

[AGWA]

Awesome - It's a proud moment for SSLMate to have its first .gov certificate in production! Thanks @konklone and @NoahKunin!

[NoahKunin]

Credit goes to your product. This is a serious advance in the ability to purchase, automate, and audit how we manage certs going forward.

On Mon, Dec 15, 2014 at 7:31 PM, Andrew Ayer notifications@github.com wrote:

Awesome - It's a proud moment for SSLMate to have its first .gov certificate in production! Thanks @konklone https://github.com/konklone and @NoahKunin https://github.com/NoahKunin!

— Reply to this email directly or view it on GitHub https://github.com/SSLMate/sslmate/issues/7#issuecomment-67093189.

Noah Kunin - Delivery Architect @noahkunin http://twitter.com/noahkunin | @18F https://twitter.com/18F

[seanherron]

(I work with @konklone and @NoahKunin)

Unfortunately, the switch to RapidSSL did not change much. I just tried to purchase a cert for another .gov, and, while it did send me over to RapidSSL for the confirmation, after confirming I got this:

Your order is pending a final quality review prior to issuance. This review is normally completed within one business day. For more information on why your order was selected for final quality review visit our FAQs at https://knowledge.geotrust.com/support/knowledge-base/index?page=content&id=SO9246

[AGWA]

Thanks for being a guinea pig with RapidSSL, @seanherron.

I've received the following from RapidSSL:

In order to manually approve the order, we need a direct contact from 18F - General Services Administration to verbally approve the order. Please provide the contact name, job title, phone number and corporate e-mail address via the contact information below.

@NoahKunin, should I pass along your contact details as found in 18F's SSLMate account? And what's your job title?

I'll be using this as a jumping off point for figuring out how to avoid this with future orders.

[seanherron]

@AGWA wow, that is kind of lame. We've already proven DNS control via MX records for email validation - I don't understand how talking to Noah over the phone provides any more security.

@NoahKunin maybe when you talk to them you can propose a longer-term solution as well?

[NoahKunin]

@seanherron Absolute.

@AGWA my info should be correct. If there needs to be a touch point between all three entities, let me know. My title is Delivery Architecture Director.

[konklone]

I wonder if direct DNS approval from a .gov domain, as discussed in #9, would make either of the 2 CAs more comfortable with enabling auto-approval of a .gov certificate.

[AGWA]

Thanks @NoahKunin - I've forwarded your details to RapidSSL. I've also reached out to Comodo. In both cases I emphasized that GSA manages the .gov TLD.

@seanherron - completely agree. This is the first time a CA has insisted on actually talking to someone, and I don't see the security benefit at all. In the past, SSLMate has only seen certificates held for manual review when the common name was similar to a domain name that's frequently targeted by phishers. In those cases, a human had to look at the website to make sure it wasn't a phishing site. This made some sense, as victims might be more likely to fall for a phishing attack if the domain name is similar and there's a padlock icon. (Of course, a savvy phisher would just swap out the website contents after the cert is approved.) I don't see the point of holding .gov domains for review when control of the domain has already been verified.

@konklone - good idea about DNS approval - I'll float the idea to them.

[NoahKunin]

Thanks @AGWA! And just to put a fine point on it, there are two authorities here:

• Management of the overall .gov space, which is done via several teams at the GSA.
• Management of specific x.gov or x.x.gov domains are managed by their various systems owners. 18F, the team within GSA we all work for, is the owner of many Federal systems on behalf of many of our Federal partners (most non-GSA, some within GSA). We ourselves are Federal staff, providing a service to the rest of Government.

Since there is no binding that is really possible between a specific function of the Government with a specific domain name, we think the two validation points of working at GSA and demonstrating technical control over the resource in question is sufficient, perhaps with some initial "onboarding" where our identity and organizational placement is validated out of band, once, and then as long as the technical requirements are met is not asked for again.

[konklone]

I don't know that working at GSA is really useful in determining whether our .gov certificates should be auto-approved. DNS-based validation by any .gov domain should get the cert auto-approved -- every government agency who wants the level of automation we're going for should get it.

[seanherron]

Agree with @konklone - DNS certification is much more secure than Noah calling up and getting verified for a cert because he happens to have access to a particular email address. That said, if DNS isn't good enough for them, perhaps the combination of DNS control and the requestor email address coming from gsa.gov would be sufficient?

[AGWA]

Comodo claims that myra.treasury.gov was held not because of .gov but because of the word "treasury," presumably for the anti-phishing reasons I described above. I will ask if this can be disabled for .gov certs since registration below the .gov TLD is obviously limited to trustworthy entities.

In the meantime, I've switched 18F back to Comodo. @seanherron, do you want to try purchasing your cert again? I think Comodo is going to be much more reasonable about this than RapidSSL, who still hasn't gotten back to me and is insisting on this stupid phone verification.

[AGWA]

I see the latest cert took 18 minutes from Comodo. I'm itching to know - was it held for manual approval that completed quickly, or was there a delay receiving/responding to the verification email?

[seanherron]

There was a delay responding to the verification email, doesn't look like there was any manual review!

[konklone]

Okay! Then I guess we really need to stick with Comodo then -- if we use RapidSSL, we'll face manual verification for all of our certs.

I think we can easily handle the occasional manual delay the first time we issue a cert for a new project that happens to trigger a phishing warning.

What would be a bigger hassle -- and prevent us from hitting the Grand Dream of ephemeral keys and short-lived automated certs -- is if every reissuance of the myra.treasury.gov certificate required an unpredictable manual review delay. I'm sure Comodo would get sick of regular manual reviews, too.

I just went and clicked the link in the approval email for the re-issuance of myra.treasury.gov (this is from Comodo), which I was using to evaluate the reissuance process in #9, and got the same manual holdup:

Do you think there's any way to mitigate the manual review process for our account for subsequent issuances of a certificate? Or if DNS-based validation would raise the level of comfort for Comodo around the review process?

[AGWA]

That's great there was no manual review this time! Comodo it will be. (BTW, I've refunded the RapidSSL order.)

One feature that could help with delays is the --temp option to sslmate buy. When used, SSLMate immediately installs a self-signed certificate, and a subsequent sslmate download will install the real cert. This is nice because it gives you a cert file you can use to immediately configure your web server, which makes automation easier and more predictable.

Manual review of reissued certs is extremely silly (though it took less than an hour this time) and I'll raise the issue with them. However, SSLMate does handle asynchronous reissues fairly well. You can use the --no-wait option to sslmate reissue and it returns immediately instead of waiting for the new cert. The new key file is saved in /etc/sslmate/example.com.key.new. A subsequent sslmate download will both download the new certificate and move the new key file into place. So, along with DNS-based validation, reissues could be automated to the extent that you wouldn't even notice if the reissue was delayed.

[konklone]

Manual review of reissued certs is extremely silly (though it took less than an hour this time) and I'll raise the issue with them.

Thank you! You've been doing a lot of manual customer support legwork for us, and I appreciate it.

You can use the --no-wait option to sslmate reissue and it returns immediately instead of waiting for the new cert. The new key file is saved in /etc/sslmate/example.com.key.new. A subsequent sslmate download will both download the new certificate and move the new key file into place. So, along with DNS-based validation, reissues could be automated to the extent that you wouldn't even notice if the reissue was delayed.

That's terrific, I did not know I could do that. I think we'll start using that. I guess it'd be tough to commit to 2-day certs (which, again, is a future thing, we'll need to iron out kinks on our way down there) with a possible 1-day delay, but we definitely have enough leg room to start automating our processes.

[konklone]

Is there anything outstanding here? We've since issued a number of certs for .gov domains, and haven't faced any more manual review.