search

zkemail / archive.prove.email

A repository to store historical, timestamped DKIM keys; and for anyone to upload their own. Basically https://archive.org for public key registries.
https://registry.prove.email
MIT License
4 stars 2 forks source link

Process key values in the database, add normalized keys to new column #85

Closed foolo closed 1 month ago

foolo commented 2 months ago

(Found when investigating non-matching keys found by the RSA GCD solver.)

Some DNS records are faulty and include none-base64 characters, which makes the key incorrect, but the key matches the results if we remove those extra characters.

Example:

dig Intel._domainkey.intel.com TXT +short

Result:

"v=DKIM1; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAoLHjLKeKOHT3Kam14Oge8XVJz0k7fcz5u7z1hrEDPWJ8
Um5ov1GVo2c4YnaDWJVkUwfaVboUxFWu8iirLvf6IEBw03ebMGItdVMhIhISlFa59EvgosmDZWCbx+OZ2dSDYNOX6n6ee6feS5qO
V2lOvMvpg6aVpHemhuADGWjyS90aAmqO7y8xOcgC0AYwp2YRVKeXayRi" " A1qPA/gFtI+M/MGFuBlrXBmOX7AwwCnpDVrwcO+N
MUHyRM8rfPUssWljRHUQRjcmd3HLn1z75AiFWMpi7eMaKyN+QjnlYHHmEpdO0OpRlFQ96dfPrDlZa2fCAyCsgT4yGDyw2NdHIXQz
9wIDAQAB;"

Note the extra space in the second partial string

We should probably clean these up before adding them to the database, and also fix existing such values in the database.

foolo commented 2 months ago

Looking a bit more at this, and realized that we probably also should make sure that every sanitized key is a valid ASN1 (DER) key. Yes, we could just remove certain characters and hope that the resulting value is a valid key value, but the keys could be broken in many ways that are hard to predict, and a hot-fix might just give more trouble.

So I would suggest the approach below, which is more work, but it would solve this and parts of other issues at the same time:

  1. First, implement https://github.com/zkemail/archive.prove.email/issues/83
  2. Update the logic so that it verifies that all new keys we add to the database are either valid ASN1 or empty. If we really want to store broken keys, we could flag them as "invalid" in the database.
  3. Batch process existing keys in the database and verify that they are either valid or empty. If not, do a new improved DNS lookup according to 1. and 2. If the keys are then still invalid, remove them, or mark as "invalid", so that we never present them to the user.
Divide-By-0 commented 2 months ago

This makes sense!

foolo commented 1 month ago

New database columns keyType and keyData for DkimRecord: https://github.com/zkemail/archive.prove.email/commit/b1a66d0bfc5ec5c25d19d620c91063ae7230ed15