store: darwin/OSX/macOS support

[adamdecaf]

Features:

• [ ] backup
• [x] list
• [ ] remove
• [ ] restore

Links:

• https://www.apple.com/certificateauthority/ca_program.html
• https://github.com/golang/go/commit/4dbcacda961abb0ca3490dc06923fa7ab344d702

Questions:

• What changes between the various versions?

[adamdecaf]

I'm stuck on decoding what the data is before it's base64 encoded in the plist format. After some whitespace mucking paired with asn1 decoding I haven't gotten anywhere. I have reached out to a couple folks who might know (or know someone who has a reference).

Resources (That haven't been very helpful)

• https://developer.apple.com/library/content/documentation/Cocoa/Conceptual/PropertyLists/UnderstandXMLPlist/UnderstandXMLPlist.html
• https://developer.apple.com/legacy/library/documentation/Darwin/Reference/ManPages/man1/security.1.html
• https://developer.apple.com/documentation/security/certificate_key_and_trust_services/trust/usage_constraints_dictionary_keys?language=objc
• https://opensource.apple.com/source/SecurityTool/
• https://github.com/unofficial-opensource-apple/Security/blob/bcd89df16fa72a2cddd20c65a934b2fa33748723/SecurityTests/clxutils/ocspTool/ocspRequest.cpp#L118
• https://github.com/search?p=2&q=plist+issuerName+&type=Code&utf8=%E2%9C%93
• https://github.com/chengr28/RevokeChinaCerts
• https://github.com/unofficial-opensource-apple/security_certificates/tree/873be7c2b427896c1bfd8e1638102207dbdd0f34
• https://github.com/ctt-gob-es/clienteafirma/blob/fbbe6b95c37307db6902b8509da43ae60db15a25/afirma-core-keystores/src/main/java/es/gob/afirma/keystores/FileBasedKeyStoreManagerFactory.java

[adamdecaf]

https://github.com/adamdecaf/cert-manage/commit/8cd05dbaf61f833d166201159bfc7eac7dc2e13e landed support for this, wow...

Here's some links for future me.

• https://www.ietf.org/rfc/rfc5280.txt
• http://luca.ntop.org/Teaching/Appunti/asn1.html

[adamdecaf]

Support for this isn't working right now. I'm having a bit of trouble properly generating the plist file for backup/restore and remove.

On restore having the entries in there defaults them to "Always Trust", but they need to be "Use System Default".

On Remove the certificates aren't being set to "Never Trust" even though the plist has the fields for "Never Trust".

[adamdecaf]

In working on #34 I found this project, which might help in applying the plist attributes.

https://github.com/wbond/oscrypto/blob/master/oscrypto/_osx/trust_list.py#L83

[adamdecaf]

What about running security remove-trusted-cert instead? I was worried about using it before, but we delete certs out of other stores (nss is a good example). We've got a separate issue to fail/warn if no backups exist prior to actually running -whitelist.

$ /usr/bin/security remove-trusted-cert
No cert file specified.
Usage: remove-trusted-cert  [-d] [-D] [certFile]
    -d                  Remove from admin cert store (default is user)
    -D                  Remove default setting instead of per-cert setting
    certFile            Certificate(s)
        Remove trusted certificate(s).

Links:

• https://github.com/author/certutil/blob/eb79cca4d3af898d556449a1371a1bddf438513c/lib/cmd/trustedroot/trust_darwin.go#L46
• https://github.com/pivotal-cf/pcfdev-cli/blob/d4f71e3ec080ef43cadfc16893266b0ad1d0d38f/cert/concrete_system_store_darwin.go#L86

[adamdecaf]

I tried first to backup/restore the *.keychain files directly, but there are some paths under /System which aren't allowing edits. Instead now I'm going to continue forward with this diff but use the security cli tool to backup/restore directly into each keychain.

git diff store/darwin.go

``` adam@~/code/src/github.com/adamdecaf/cert-manage$ git diff store/darwin.go | tee diff --git a/store/darwin.go b/store/darwin.go index ef4cb1a..06c76c6 100644 --- a/store/darwin.go +++ b/store/darwin.go @@ -57,25 +57,38 @@ func platform() Store { } // Backup will save off a copy of the existing trust policy +// TODO(adam): NEW(-NEW) PLAN +// Backup(): Export all certs from each keychain file, store under darwin/$time/$keychain-name/$fingerprint.crt +// Remove(): Iterate over and call remove-trusted-cert on each certificate not matching whitelist (probably already done) +// Restore(): Iterate over latest backup dir calling add-trusted-cert foreach certificate func (s darwinStore) Backup() error { - fd, err := trustSettingsExport() + // setup (and create) backup dir + dir, err := getCertManageDir(fmt.Sprintf("%s/%d", darwinBackupDir, time.Now().Unix())) if err != nil { return err } - defer os.Remove(fd.Name()) - // Copy the temp file somewhere safer - outDir, err := getCertManageDir(darwinBackupDir) + // copy each keychain file into the backup dir + chains, err := getKeychainPaths(systemKeychains) if err != nil { return err } - filename := fmt.Sprintf("trust-backup-%d.xml", time.Now().Unix()) - out := filepath.Join(outDir, filename) - - // Copy file - err = file.CopyFile(fd.Name(), out) + for i := range chains { + if _, err := os.Stat(chains[i]); os.IsNotExist(err) { + if debug { + fmt.Printf("store/darwin: skipping %s because it does not exist\n", chains[i]) + } + continue + } - return err + // Backup keychain file + _, fname := filepath.Split(chains[i]) + err = file.CopyFile(chains[i], filepath.Join(dir, fname)) + if err != nil { + return err + } + } + return nil } // List @@ -83,11 +96,11 @@ func (s darwinStore) Backup() error { // Note: Currently we are ignoring the login keychain. This is done because those certs are // typically modified by the user (or an application the user trusts). func (s darwinStore) List() ([]*x509.Certificate, error) { - uchains, err := getUserKeychainPaths() + chains, err := getKeychainPaths(systemKeychains) if err != nil { return nil, err } - installed, err := readInstalledCerts(append(systemKeychains, uchains...)...) + installed, err := readInstalledCerts(chains...) if err != nil { return nil, err } @@ -264,91 +277,93 @@ func (s darwinStore) Remove(wh whitelist.Whitelist) error { return err } - // Keep what's whitelisted - kept := make([]*x509.Certificate, 0) - for i := range certs { - if wh.Matches(certs[i]) { - kept = append(kept, certs[i]) + remove := func(cert *x509.Certificate) error { + tmp, err := ioutil.TempFile("", "cert-manage-remove-trusted-cert") + if err != nil { + return err + } + defer os.Remove(tmp.Name()) + + // shell out to remove-trusted-cert + cmd := exec.Command("/usr/bin/security", "remove-trusted-cert", "-d", tmp.Name()) + out, err := cmd.CombinedOutput() + if debug { + fp := _x509.GetHexSHA256Fingerprint(*cert) + if err != nil { + fmt.Printf("ERROR: during remove-trusted-cert (fingerprint=%s), error=%v\n", fp, err) + } else { + fmt.Printf("remove-trusted-cert (for %s)\n%s\n", fp, out) + } } + return err } - // Build plist xml file and restore on the system - items := make(trustItems, 0) - for i := range kept { - if kept[i] == nil { - continue + // Remove certificates that are not whitelisted + for i := range certs { + if !wh.Matches(certs[i]) { + err := remove(certs[i]) + if err != nil { + return err + } } - items = append(items, trustItemFromCertificate(*kept[i])) } + return nil +} + +func (s darwinStore) Restore(where string) error { + // TODO(adam): We should just take `where` if it exists. + // If it's a dir, process like usual, if it's a file then + // only replace that specific file. - // Create temporary output file - f, err := ioutil.TempFile("", "cert-manage") + // Find latest backup dir + dir, err := getCertManageDir(darwinBackupDir) if err != nil { return err } - // show plist file if we're in debug mode, otherwise cleanup - if debug { - fmt.Printf("darwin.Remove() plist file: %s\n", f.Name()) - } else { - defer os.Remove(f.Name()) + dir, err = getLatestBackupFile(dir) + if err != nil { + return err } - // Write out plist file - // TODO(adam): This needs to have set the trust settings (to Never Trust), the fields lower on - // https://github.com/ntkme/security-trust-settings-tools/blob/master/security-trust-settings-blacklist/main.m#L10 - err = items.toXmlFile(f.Name()) + // Grab each keychain file and put it back, if filenames match up + fds, err := ioutil.ReadDir(dir) if err != nil { return err } - return s.Restore(f.Name()) -} - -// TODO(adam): This should default trust to "Use System Trust", not "Always Trust" -// Maybe this is a change for "Backup"...? -func (s darwinStore) Restore(where string) error { - // Setup file to use as restore point - if where == "" { - dir, err := getCertManageDir(darwinBackupDir) - if err != nil { - return err - } - - // Ignore any errors and try to set a file - latest, _ := getLatestBackupFile(dir) - where = latest - } - if where == "" { - // No backup dir (or backup files) and no -file specified - return errors.New("No backup file found and -file not specified") - } - if !file.Exists(where) { - return errors.New("Restore file doesn't exist") + // Grab known keychains, used for matching and targets for overwritable + // files. + chains, err := getKeychainPaths(systemKeychains) + if err != nil { + return err } - - // run restore - args := []string{"/usr/bin/security", "trust-settings-import", "-d", where} - cmd := exec.Command("sudo", args...) - out, err := cmd.CombinedOutput() - - if err != nil && debug { - fmt.Printf("Command ran: '%s'\n", strings.Join(cmd.Args, " ")) - fmt.Printf("Output was: %s\n", string(out)) + for i := range fds { // each file to recover + for j := range chains { // each possible overwrite point + _, name1 := filepath.Split(fds[i].Name()) + _, name2 := filepath.Split(chains[j]) + if name1 == name2 { // matched, "login.keychain" == "login.keychain" + src := filepath.Join(dir, fds[i].Name()) + err = file.SudoCopyFile(src, chains[j]) + if err != nil { + return err + } + break + } + } } - - return err + return nil } -func getUserKeychainPaths() ([]string, error) { +func getKeychainPaths(initial []string) ([]string, error) { uhome := file.HomeDir() if uhome == "" { return nil, errors.New("unable to find user's home dir") } - return []string{ + return append(systemKeychains, filepath.Join(uhome, "/Library/Keychains/login.keychain"), filepath.Join(uhome, "/Library/Keychains/login.keychain-db"), - }, nil + ), nil } // trustItems wraps up a collection of trustItems parsed from the `security` cli tool ```

[adamdecaf]

Figured it out. We can just modify the login keychain. There's fs protection (overridable with a restart and commands), but using the login keychain is much safer as it's applying deny overrides.

// adam@~/code/src/github.com/adamdecaf/cert-manage$ security add-trusted-cert -d -r deny -k ~/Library/Keychains/login.keychain DE28F4A4FFE5B92FA3C503D1A349A7F9962A8212.crt
// adam@~/code/src/github.com/adamdecaf/cert-manage$ sudo security delete-certificate -t -Z DE28F4A4FFE5B92FA3C503D1A349A7F9962A8212 ~/Library/Keychains/login.keychain
//
// Verified with (fails with "SSL certificate problem: Invalid certificate chain"), requires -k override
// curl -v -L -I https://google.com

Links

• https://superuser.com/questions/1070664/security-seckeychainitemdelete-unixoperation-not-permitted-on-os-x-when-tryi
• https://apple.stackexchange.com/questions/24640/how-do-i-remove-many-system-roots-from-apple-system-keychain

(For reference, here's some examples that don't work.)

// Write cert in DER format to tempfile
// err = der.ToFile(tmp.Name(), cert)
// if err != nil {
//  return err
// }

// TODO(adam): What about 'delete-certificate' paired with 'add-trusted-cert -r deny' ??
// root@~# security add-trusted-cert -r deny /Users/adam/code/src/github.com/adamdecaf/cert-manage/cert.pem
// root@~# security delete-certificate -c 'GeoTrust Primary Certification Authority - G2' // TODO(adam): Use -Z <sha1-fingerprint> instead

// cmd := exec.Command("/usr/bin/security", "remove-trusted-cert", "-d", tmp.Name()) // TODO(adam): -d vs <nothing>, w/ -D can't specify certFile
// fp := _x509.GetHexSHA1Fingerprint(*cert)
// cmd := exec.Command("/usr/bin/security", "add-trusted-cert", "-r", "deny", tmp.Name())
// cmd := exec.Command("/usr/bin/security", "delete-certificate", "-Z", fp)

[adamdecaf]

I was reading about security authorize and security authorizationdb which might be closer to how we can automate cert install/modify. I think we can get some sort of session initialized and then call add-trusted-cert -r deny several times, but maybe not.

https://developer.apple.com/legacy/library/documentation/Darwin/Reference/ManPages/man1/security.1.html https://derflounder.wordpress.com/2014/02/16/managing-the-authorization-database-in-os-x-mavericks/

authorizationdb looks interesting and might offer a way to temporarily override trust so the user can apply a batch of trust settings, but maybe only on 10.13+

https://www.dssw.co.uk/blog/2013-10-26-authorization-rights-and-mavericks/ https://www.dssw.co.uk/reference/authorization-rights/index.html https://www.dssw.co.uk/reference/authorization-rights/system-identity-write-credential.html https://www.dssw.co.uk/reference/authorization-rights/system-identity-write-self.html https://www.dssw.co.uk/reference/authorization-rights/system-keychain-modify.html https://www.dssw.co.uk/reference/authorization-rights/system-keychain-create-loginkc.html https://www.dssw.co.uk/reference/authorization-rights/com-apple-configurationprofiles-userprofile-trustcert.html https://github.com/facebook/osquery/pull/1318

I also found this macenterprise mailing list, which I've applied for, but haven't gotten a response back yet.

https://groups.google.com/forum/#!searchin/macenterprise/keychain$20password|sort:date/macenterprise/me5WFUgu6wc/6ewkISdvDAAJ https://groups.google.com/forum/#!searchin/macenterprise/keychain$20password|sort:date/macenterprise/1S2O1IbQg7w/ypAD6aa-BAAJ https://groups.google.com/forum/#!searchin/macenterprise/%22add-trusted-cert%22%7Csort:date/macenterprise/u8d8yHjuYSQ/VJnjDBXBN1EJ

https://stackoverflow.com/questions/9837784/how-do-you-write-to-the-osx-system-keychain/27264534#27264534 https://stackoverflow.com/questions/4362723/any-idea-to-delete-item-in-system-keychain-in-command-or-applescript https://opensource.apple.com/source/Security/Security-57740.51.3/OSX/authd/

[adamdecaf]

Pretty sure I've got this now.

$ # distrust
$ sudo security add-trusted-cert -d -r deny -p ssl -k /Library/Keychains/System.keychain aaa.pem

$ # restore trust
$ sudo security add-trusted-cert -d -r unspecified -k /Library/Keychains/System.keychain aaa.pem

$ # cleanup System.keychain
$ sudo security delete-certificate -Z D1EB23A46D17D68FD92564C2F1F1601764D8E349 /Library/Keychains/System.keychain

[adamdecaf]

The whitelisting/removal process is incomplete. I need to survey the proper plist to generate and probably just use those (with trust-settings-import) via Go's templating. The security cli tool doesn't appear to give us the flexibility we want and I think I've got the plist reverse engineered enough to move forward.

[adamdecaf]

Actually, what about hooking into the C / Security api?

https://github.com/docker/docker-credential-helpers/blob/master/osxkeychain/osxkeychain_darwin.go https://github.com/docker/docker-credential-helpers/blob/master/osxkeychain/osxkeychain_darwin.c

Or using the go-plist library: https://github.com/DHowett/go-plist

[adamdecaf]

https://github.com/keybase/go-keychain

[RaheelaKhan1172]

This is super intense and good luck 🍀 !!

I want to learn Golang now...

[adamdecaf]

https://github.com/FiloSottile/mkcert/blob/master/truststore_darwin.go