[WIP]Discussion: x.509 Certificates - deterministic handling and fingerprinting.

[WIP] Issue

Summary

This issue is intended to serve as a point of discussion related to the effort to standardize our handling of the distinguished name values found in x.509 certificate subject and issuer fields. The intent is to arrive at a field ordering that is

portable across programming languages (at least Java, Python, and Go)
deterministic even when processing unknown fields

Commentary from HD about handling in Go:

https://github.com/rapid7/recog/blob/94c2576bd8c91365a4d52f21e551e466b7b2600f/xml/x509_subjects.xml#L4-L38

Commentary and example from HD on handling of extra fields in Go:

The relevant Go code for extra name handling:

          oidString := tv.Type.String()
          typeName, ok := attributeTypeNames[oidString]
          if !ok {
              derBytes, err := asn1.Marshal(tv.Value)
              if err == nil {
                  s += oidString + "=#" + hex.EncodeToString(derBytes)
                  continue // No value escaping necessary.
              }

              typeName = oidString
          }

In practice this looks like:

CN=device.corp.com,OU=VMware Engineering,O=VMware,L=Palo Alto,ST=California,C=US,1.2.840.113549.1.9.1=#0c0f766d636140766d776172652e636f6d

1.2.840.113549.1.9.1 is the OID of the EMAILADDRESS attribute. The value includes the DER bytes, including the Type and Length before the value (vmca@vmware.com).

TODO before requesting feedback from others:

[ ] Build simple Python, Ruby, Go, and Rust tools that connect to an HTTP endpoint and emit the certificate with the least amount of processing
[ ] Use the tools above to provide reference examples to show the differences

Examples

Some example subject values that haven't been accounted for in our prior discussions.

unstructuredName

{
  "C": "US",
  "CN": "foo.bar",
  "L": "Palo Alto",
  "O": "VMware, Inc",
  "OU": "VMware ESX Server Default Certificate",
  "ST": "California",
  "emailAddress": "ssl-certificates@vmware.com",
  "unstructuredName": "1617207215,564d7761726520496e632e"
}

businessCategory, jurisdictionC , jurisdictionL, jurisdictionST

{
  "C": "DE",
  "CN": "foo.bar",
  "L": "Hannover",
  "O": "TUI AG",
  "ST": "Niedersachsen",
  "businessCategory": "Private Organization",
  "jurisdictionC": "DE",
  "jurisdictionL": "Charlottenburg",
  "jurisdictionST": "Berlin",
  "serialNumber": "HRB 000"
}

name

{
  "name": "AR2220-Self-Signed-Certificate-2102352934DMGA000458"
}

description

{
  "CN": "00:09:52:05:b9:73",
  "O": "Auerswald",
  "description": "Vendor=Auerswald;SN=0000000000;MAC=00:00:00:00:00:00;DevClass=PBX;DevTyp=COMpact5200"
}`

From Microsoft Active Directory, DC

{
  "CN": "foo.bar.local",
  "DC": "va",
  "OU": "devices"
}

rapid7 / recog