search

notaryproject / notary

Notary is a project that allows anyone to have trust over arbitrary collections of data
Apache License 2.0
3.22k stars 508 forks source link

Purging GUN delegation keys for a role prevents further trust operations due to threshold #1500

Open tim-gp opened 5 years ago

tim-gp commented 5 years ago

I can't find a way to either revoke a compromised delegate, or replace and remove an expired delegate. I'm hoping this is user error on my part.

I originally posted as a comment here: https://github.com/theupdateframework/notary/issues/1444#issuecomment-503549414 but am creating a standalone issue in the hope of a response.

Here are the steps to reproduce using the notary compose file (from master) and a locally-running registry:

# start notary and a registry
docker-compose up -d
docker run -d -p 5000:5000 --restart=always --name registry registry:2

# create a root key, initialise GUN and rotate snapshot key
notary key generate
notary -s https://localhost:4443 --tlscacert fixtures/root-ca.crt init localhost:5000/busybox -p 
notary -s https://localhost:4443 --tlscacert fixtures/root-ca.crt key rotate localhost:5000/busybox snapshot -r 

# generate two private and public keys, chmod private keys to 600 so docker can load
openssl genrsa -out delegate1.key 2048
openssl rsa -pubout -in delegate1.key -out delegate1.crt
openssl genrsa -out delegate2.key 2048
openssl rsa -pubout -in delegate2.key -out delegate2.crt
chmod 600 delegate*.key

# add the first delegate key
notary -s https://localhost:4443 --tlscacert fixtures/root-ca.crt delegation add localhost:5000/busybox targets/releases delegate1.crt --all-paths -p

# setup DCT
docker pull busybox
docker tag busybox localhost:5000/busybox:latest
export DOCKER_CONTENT_TRUST=1
export DOCKER_CONTENT_TRUST_SERVER=https://localhost:4443
docker push localhost:5000/busybox:latest # fails 'no valid signing keys for delegation roles'

# load first delegate key into docker
docker trust key load delegate1.key
docker push localhost:5000/busybox:latest # succeeds

# add second delegate key to notary
notary -s https://localhost:4443 --tlscacert fixtures/root-ca.crt delegation add localhost:5000/busybox targets/releases delegate2.crt --all-paths -p

# sign with first key
docker push localhost:5000/busybox:latest # succeeds

# remove first key from docker and load second key
rm ~/.docker/trust/private/*
docker trust key load delegate2.key

# sign with second key
docker push localhost:5000/busybox:latest # succeeds

# remove first key from notary, then list delegations
notary -s https://localhost:4443 --tlscacert fixtures/root-ca.crt delegation purge localhost:5000/busybox --key <firstkeyid> -p
notary -s https://localhost:4443 --tlscacert fixtures/root-ca.crt delegation list localhost:5000/busybox # shows WARN[0000] Error getting targets/releases: valid signatures did not meet threshold for targets/releases

docker push localhost:5000/busybox:latest # fails 'WARN[0000] Error getting targets/releases: valid signatures did not meet threshold for targets/releases - failed to sign localhost:5000/busybox:latest: trust server rejected operation.'

Here is the debug output from delegation list after purging the first key:

notary -D -s https://localhost:4443 --tlscacert fixtures/root-ca.crt delegation list localhost:5000/busybox
DEBU[0000] Configuration file not found, using defaults 
DEBU[0000] Using the following trust directory: /Users/timgp/.notary 
DEBU[0000] No yubikey found, using alternative key storage: no library found 
DEBU[0000] Making dir path: /Users/timgp/.notary/tuf/localhost:5000/busybox/changelist 
DEBU[0000] entered ValidateRoot with dns: localhost:5000/busybox 
DEBU[0000] found the following root keys: [ee39a8b99440b38fc3df04ec51bac125bbab442f4498b84bb1a2f64a8c49c741] 
DEBU[0000] found 1 valid leaf certificates for localhost:5000/busybox: ee39a8b99440b38fc3df04ec51bac125bbab442f4498b84bb1a2f64a8c49c741 
DEBU[0000] found 1 leaf certs, of which 1 are valid leaf certs for localhost:5000/busybox 
DEBU[0000] checking root against trust_pinning config for localhost:5000/busybox 
DEBU[0000] checking trust-pinning for cert: ee39a8b99440b38fc3df04ec51bac125bbab442f4498b84bb1a2f64a8c49c741 
DEBU[0000]  role has key IDs: ee39a8b99440b38fc3df04ec51bac125bbab442f4498b84bb1a2f64a8c49c741 
DEBU[0000] verifying signature for key ID: ee39a8b99440b38fc3df04ec51bac125bbab442f4498b84bb1a2f64a8c49c741 
DEBU[0000] root validation succeeded for localhost:5000/busybox 
DEBU[0000] entered ValidateRoot with dns: localhost:5000/busybox 
DEBU[0000] found the following root keys: [ee39a8b99440b38fc3df04ec51bac125bbab442f4498b84bb1a2f64a8c49c741] 
DEBU[0000] found 1 valid leaf certificates for localhost:5000/busybox: ee39a8b99440b38fc3df04ec51bac125bbab442f4498b84bb1a2f64a8c49c741 
DEBU[0000] found 1 leaf certs, of which 1 are valid leaf certs for localhost:5000/busybox 
DEBU[0000] checking root against trust_pinning config for localhost:5000/busybox 
DEBU[0000] checking trust-pinning for cert: ee39a8b99440b38fc3df04ec51bac125bbab442f4498b84bb1a2f64a8c49c741 
DEBU[0000]  role has key IDs: ee39a8b99440b38fc3df04ec51bac125bbab442f4498b84bb1a2f64a8c49c741 
DEBU[0000] verifying signature for key ID: ee39a8b99440b38fc3df04ec51bac125bbab442f4498b84bb1a2f64a8c49c741 
DEBU[0000] root validation succeeded for localhost:5000/busybox 
DEBU[0000] updating TUF client                          
DEBU[0000] Loading timestamp...                         
DEBU[0000] 200 when retrieving metadata for timestamp   
DEBU[0000] timestamp role has key IDs: 6b959d30512cabb4e70fcb0cf2a66db478c0875b7b6a3aa50974eeb669848ae4 
DEBU[0000] verifying signature for key ID: 6b959d30512cabb4e70fcb0cf2a66db478c0875b7b6a3aa50974eeb669848ae4 
DEBU[0000] timestamp role has key IDs: 6b959d30512cabb4e70fcb0cf2a66db478c0875b7b6a3aa50974eeb669848ae4 
DEBU[0000] verifying signature for key ID: 6b959d30512cabb4e70fcb0cf2a66db478c0875b7b6a3aa50974eeb669848ae4 
DEBU[0000] successfully verified downloaded timestamp   
DEBU[0000] Loading snapshot...                          
DEBU[0000] snapshot role has key IDs: 62bceb20b28ce8b0005a31dda69d54bf61022d9ee53eb7104e45dbd15093004a 
DEBU[0000] verifying signature for key ID: 62bceb20b28ce8b0005a31dda69d54bf61022d9ee53eb7104e45dbd15093004a 
DEBU[0000] successfully verified cached snapshot        
DEBU[0000] Loading targets...                           
DEBU[0000] targets role has key IDs: 71a043b9200cf32c4bb7fa056082b37c8e43fe3869b51c43b12ea81e55abf7b9 
DEBU[0000] verifying signature for key ID: 71a043b9200cf32c4bb7fa056082b37c8e43fe3869b51c43b12ea81e55abf7b9 
DEBU[0000] successfully verified cached targets         
DEBU[0000] Loading targets/releases...                  
DEBU[0000] targets/releases role has key IDs: 363f90a2df620cc9e7b79bed87aa8b66489697c4e312608376f4f8a1abef918d 
DEBU[0000] verifying signature for key ID: cb1f6014718e5c2346f887c60523f38bc4967ce9fe8f1689c70cd08b73a5613b 
DEBU[0000] continuing b/c keyid lookup was nil: cb1f6014718e5c2346f887c60523f38bc4967ce9fe8f1689c70cd08b73a5613b
DEBU[0000] cached targets/releases is invalid (must download): valid signatures did not meet threshold for targets/releases 
DEBU[0000] 200 when retrieving metadata for targets/releases.9691d93dcd84e356822136ec58d5e96ea5b428d5b39a710f044e07be35b94f96 
DEBU[0000] targets/releases role has key IDs: 363f90a2df620cc9e7b79bed87aa8b66489697c4e312608376f4f8a1abef918d 
DEBU[0000] verifying signature for key ID: cb1f6014718e5c2346f887c60523f38bc4967ce9fe8f1689c70cd08b73a5613b 
DEBU[0000] continuing b/c keyid lookup was nil: cb1f6014718e5c2346f887c60523f38bc4967ce9fe8f1689c70cd08b73a5613b

DEBU[0000] downloaded targets/releases.9691d93dcd84e356822136ec58d5e96ea5b428d5b39a710f044e07be35b94f96 is invalid: valid signatures did not meet threshold for targets/releases 
WARN[0000] Error getting targets/releases: valid signatures did not meet threshold for targets/releases 

ROLE                PATHS             KEY IDS                                                             THRESHOLD
----                -----             -------                                                             ---------
targets/releases    "" <all paths>    363f90a2df620cc9e7b79bed87aa8b66489697c4e312608376f4f8a1abef918d    1

The interesting line is DEBU[0000] verifying signature for key ID: cb1f6014718e5c2346f887c60523f38bc4967ce9fe8f1689c70cd08b73a5613b - in my example the first delegate key was id cb1... and the second was id 363.... It appears that the targets/releases role may still be associated with the first delegate key.

When I re-add the first public key with delegation add the threshold messages stop and I can continue to push and sign content.

I would really like to understand how delegation keys can be updated in the case of either certificate expiry or key compromise as I cannot see how it works in practice.

dnwake commented 5 years ago

I am seeing a related issue (https://github.com/theupdateframework/notary/issues/1499)

We created delegations in order to allow a large number of machines to push to the same images without worrying about having the signing key on each one. It looks like the default expiration date for delegations is 3 years (or it was when we started created them around 3 years ago). This is now hitting us ...