Generating RFC 5280 conformant serial numbers is slightly treacherous.
Section 4.1.2.2 dictates that conforming implementations "MUST NOT use
serialNumber values longer than 20 octets". A seemingly obvious way to
pick serials that conform to this requirement is choosing a random int
between [0, 1 << 20*8), which is what the mitm package previously did.
The pitfall here is that the DER encoding of integers uses the MSB to
indicate the sign, so if encoding/asn1 is passed a positive big.Int for
which the encoding has the MSB set it will prefix the encoding with a
0x00 byte. The result of this is that if a serial number is picked that
is exactly 20 bytes, and has its MSB set, it will be encoded as 21
bytes.
The simple solution is to just re-generate the serial if you happen to
pick one which is exactly 20 bytes with the MSB set. Since there are a
number of users of mitm.MaxSerialNumber (both within this project, and
externally), a helper function is added that can be used as a drop-in
replacement for the existing crypto/rand.Int calls.
This came up because we attempted to add a restriction to Go's crypto/x509.CreateCertificate
which limited encoded serial numbers to 20 octets, which broke a number of tests inside of
Google, a handful of which relied on this project. Googlers can see b/229601555 for some
further context here.
Generating RFC 5280 conformant serial numbers is slightly treacherous. Section 4.1.2.2 dictates that conforming implementations "MUST NOT use serialNumber values longer than 20 octets". A seemingly obvious way to pick serials that conform to this requirement is choosing a random int between [0, 1 << 20*8), which is what the mitm package previously did. The pitfall here is that the DER encoding of integers uses the MSB to indicate the sign, so if encoding/asn1 is passed a positive big.Int for which the encoding has the MSB set it will prefix the encoding with a 0x00 byte. The result of this is that if a serial number is picked that is exactly 20 bytes, and has its MSB set, it will be encoded as 21 bytes.
The simple solution is to just re-generate the serial if you happen to pick one which is exactly 20 bytes with the MSB set. Since there are a number of users of mitm.MaxSerialNumber (both within this project, and externally), a helper function is added that can be used as a drop-in replacement for the existing crypto/rand.Int calls.
This came up because we attempted to add a restriction to Go's crypto/x509.CreateCertificate which limited encoded serial numbers to 20 octets, which broke a number of tests inside of Google, a handful of which relied on this project. Googlers can see b/229601555 for some further context here.