SCT self-consistency lint

[jsha]

In following up on https://bugzilla.mozilla.org/show_bug.cgi?id=1838667 and https://www.agwa.name/blog/post/last_weeks_lets_encrypt_downtime, I'd like to try contributing a configurable lint that checks a final certificate's SCTs are self-consistent: that they correspond to the TBSCertificate that results from re-encoding the final certificate's contents minus the SCT List extension.

This would require being configured with a copy of all_logs_list.json, in order to valid signatures against logs' public keys.

One way to implement this would be to pull in certificate-transparency-go as a dependency. But I'm assuming it's desirable to minimize external dependencies, so will try to implement it using just x/crypto/cryptobyte. Let me know if that assumption is wrong in either way, or if you see any issues with the proposal. Thanks!

[christopher-henderson]

Howdy @jsha!

Just to paraphrase my understanding of what we would like to accomplish, in somewhat pseudo code....

// Some lint for checking SCT sigantures
type SCTLint struct {
    confg SCTConfig
}

// A config for the lint that tells us where to get the list of logs from.
type SCTConfig struct {
    // One option is to read the keys from the FS
    File *string
    // Or reach out to the open internet to get a list.
    Url *string
}

// Let the ZLInt infrastructure configure that lint configuration.
func (s *SCTLint) Configure() interface{} {
    return &s.config
}

// If the cert has an SCT and the user has declared SOME 
// external resource  to reach out to, then continue on with the lint.
func (s *SCTLint) CheckApplies(c *x509.Certificate) bool {
    return util.HasSCT(c) && (s.config.file != nil || s.config.url != nil)
}

// The lint itself
func (s *SCTLint) Execute(c *x509.Certificate) *lint.LintResult {
    ...get the list of logs and their keys
        ...find the right issuer
        ...check the SCT signature against the key found above
}

Which then would have a configuration something like....

[SCTLint]
# For pointing to the FS
File = "..."
# For pointing to an online list
Url = "..."

Is this a fair understanding?

But I'm assuming it's desirable to minimize external dependencies

Admirable, although not mandatory. However, I reckon that we're just parsing out RFC 6962 Section 3.2 so going dependency free shouldn't be too onerous.

[aarongable]

Yep, I think this exactly matches my understanding of what it would look like, with one minor tweak: the pseudocode inside Execute() will be in a loop to cover the likely-multiple SCTs within the cert.

Also, I forget how the "config sharing" thing works, but it might be useful to ensure that this is a shareable config, so that another lint for Google's requirement of "the SCTs must come from at least two different log operators" can be added later.

[christopher-henderson]

Howdy @jsha and @aarongable

I've been quite swamped with working on a massive infusion of new certs into the test corpus for an upcoming SMIME linting feature.

Were there expectations that I open up a PR for this lint, or would the folks at LE amenable to a contribution?

[jsha]

Yep, we're planning to make a contribution. I've got working code at GitHub.com/jsha/corr. I just need to wire it up to config inputs. It turns out it needs both the known logs list and the CCADB as config inputs, since the SCT signature is computed over the issuer's subjectpublickeyinfo.

[cardonator]

This is definitely an admirable Lint. I'm trying to understand what the knock-on effects of the configuration are, especially when it comes to ICAs registered in CCADB. Would that make this lint dependent on CCADB itself (not ideal) or would we have to have a way to export information from CCADB?

Coming at this from a CA business perspective, it seems like it would be ideal for the "known hierarchies" list to be something that can be updated on a schedule easily and read by zlint, and it also seems like for the majority of certs a CA would be testing, they would know their own hierarchies without referencing CCADB.

[christopher-henderson]

Would that make this lint dependent on CCADB itself (not ideal) or would we have to have a way to export information from CCADB?

At least with regard to possible "splash damage" concerns (I.E. a lint potentially doing a large download that others may not be interested in) I would definitely suggest that the lint inspect it's configuration and simply skip itself if a resource location for the CCADB was not found.

[mcpherrinm]

The CCADB is one way to have a set of issuers that could be used, but wouldn't work well for test setups, non-public CAs, etc.

what if Zlint takes a more direct approach: Add a new function LintCertificateWithIssuer(c x509.Certificate, issuer x509.Certificate) and corresponding CLI flag.

There are other lints, like ensuring AKID/SKID match properly, which could be useful. The issuer is usually available in contexts you'd call zlint (in a CA, it should be known apriori. In a TLS scanner, you've probably done path building). That outsources the problem to some degree, but avoids the problem of needing a complete set of potential intermediates from something like CCADB

[cardonator]

There was a proposal for something like that at one time, maybe @christopher-henderson remembers what happened with that. As I recall one pre-requisite toward a backwards compatible approach was the configurable lints. That exists now, so maybe it's more of an option?

[christopher-henderson]

@mcpherrinm @cardonator

Right, the solution was configurable lints itself.

The impetus for implementing the notion of a configuration was to avoid having to completely rework the framework of parsing CLI flags and conditionally passing on that information only to lints who are interested in it. That is, it allows individual lints to essentially completely bypass having to request framework changes so that they can unilaterally do tricky maneuvers if the need arises.

This particular discussion is a great example of this - no extant lints have a particular interest in the certificate chain, so it is quite unattractive to do a risky/time consuming framework change in order to accommodate this one lint. Instead, the lint can simply declare…

[MyOneLint]
# File path, url, inline string, or whatever.
issuer = "..."

The configuration framework is flexible, however, so if we believed that the issuer would be of significance to many lints then we fortunately do have the concept of scoping in our configuration setup. That is, the issuer could be scoped to an individual lint class (E.G. RFC 5280) or even globally.

[mcpherrinm]

But the issuer differs per-certificate, so you’d have to know them all up-front, which doesn’t seem ideal in many circumstances (though is probably fine for Let’s Encrypt use, since we have to deploy to add new intermediates already)

[cardonator]

I think it is a relevant question for sure. Configurable lints solved one aspect of this, but the other is how can I dynamically pass the relevant chain through when I am linting a certificate for some of these lints? That sounds like a dynamic configuration based on the implementation, but I still agree fundamentally that it's ideal to build that outside of the expected contract of the library.

[christopher-henderson]

@mcpherrinm

The issuer differs per certificate, but each certificate is one run of the ZLint tool (each of which can absolutely have it's own configuration).

Considering that the tool is quite usually ran programmatically, I don't see any obvious issue in generating a valid toml file if one sincerely needs a different configuration for each certificate.

Restated, these configurations are not necessarily like "traditional" configurations where you have one file that stays static throughout the lifetime of the tool. It was absolutely built with the intention that CAs may generate them programmatically per certificate in order to fulfill such use cases.

[aarongable]

In high-volume environments, we don't run the zlint tool from scratch separately for each certificate -- we instantiate it as a library once at process startup, construct a registry of lints, and then re-use that registry for every pre-issuance lint cycle we do.

That said, we do use a different instance of the linter for each issuer certificate, so each instance of the in-memory linter could be configured with a different issuer certificate.