search

theupdateframework / python-tuf

Python reference implementation of The Update Framework (TUF)
https://theupdateframework.com/
Apache License 2.0
1.64k stars 273 forks source link

Public Log Proposal #340

Closed ecordell closed 6 years ago

ecordell commented 8 years ago

This proposes a change to the TUF spec to support a tamper-evident public log of changes to the root authority in a TUF repository and to allow trust-pinning for delegated roles.

There is further discussion of this idea at https://github.com/docker/notary/issues/458 and https://github.com/docker/notary/issues/835

Motivation

From the TUF spec:

To replace a compromised root key or any other top-level role key, the root role signs a new root.json file that lists the updated trusted keys for the role. When replacing root keys, an application will sign the new root.json file with both the new and old root keys until all clients are known to have obtained the new root.json file (a safe assumption is that this will be a very long time or never).

Currently, rotating the root keys requires signing with old keys indefinitely. Although this does successfully allow rotation, supporting even moderately aggressive rotation schemes (say, having a root threshold of 2/3 available keys and rotating those keys quarterly) results in a large number of old keys that need to be kept around.

Note that this may be less of a concern in TUF as-specified:

Periodically, the software update system using the framework instructs the framework to check each repository for updates.

If clients can be assumed to be constantly polling, there may be an opportunity to remove older keys. But this is not the case for package managers in general - current partial/full implementations in PyPI, RubyGems, and Notary do not have "periodic" polling as one of their properties.

Proposal

The canonical (server) TUF metadata repository should version the root.json file any time root keys change, and should point to a "parent" root.json.

{
    "signed": {
        "_type": "Root",
        // ...
        "parent": "123"  // sha256 of previous root.json
    },
    "parentSignatures" {
        // current metadata signed with parent keys/threshold
    },
    "signatures": [
        // current metadata signed with current keys/threshold
    ]
}

When a client gets new metadata with a parent and parentSignatures block, it does the following:

Rotating a single root key from A to B

Trust pinning is unspecified in TUF, though it does exist in Notary, which requires pinning an x509 cert or CA.

Using a public log on delegations would allow trust pinning with key rotation. Instead of pinning to a CA or cert, a pubic key (ideally, a set of public keys) would be pinned.

If trust is pinned for a delegation, we require that there is a valid parent chain back to the pinned public keys. This works analagously to the root key rotation case.

An interesting property that falls out of this is that users can decide where to root their trust (the registry or any delegated target owner) without losing the TUF freshness/rotation ability.

We leave the semantics of specifying pinned trust unspecified for now.

Example

Rotating a pinned delegation key

endophage commented 8 years ago

One thing that jumps out, the breakup of parentSignatures and signatures is undesirable. Whenever I get a new root, regardless of whether a root rotation has actually happened or not, I must validate it using both the root keys I know, and the root keys it contains. In many instances it will not be the root key that has changed, so parentSignatures and signatures will contain duplicate signatures for the same set of root keys.

If a root key rotation has occurred, I can't see any benefit in breaking up the signatures of the old and new keys as if there is a threshold n with n > 1, I may have rotated only one key, yet parentSignatures and signatures will now both have to contain n signatures, but with subsets of size n-1 being identical.

ecordell commented 8 years ago

Correct, there's no explicit reason they need to be separate. If we're prioritizing file size I think it makes sense to combine them (i.e. just have signatures matching the current and previous keys all in a single signatures block)

endophage commented 8 years ago

Even if we're not considering filesize, it just feels like an unnecessary complication in an already complex system, and that to me is an even more important prioritization.

ecordell commented 8 years ago

Agreed. Updated proposal reflecting that:

{
    "signed": {
        "_type": "Root",
        // ...
        "parent": "123"  // sha256 of previous root.json
    },
    "signatures": [
        // current metadata signed with current keys/threshold
        // current metadata signed with parent keys/threshold
    ]
}
JustinCappos commented 8 years ago

Forgive the clarifying question but I'm a little confused by this and want to be sure we are on the same page before saying anything substantive.

I think the issue you want to solve is efficiently getting a chain of trust from the oldest known to current piece of root metadata. If you have 3 keys, and want to rotate one out every hour, one could simply sign the new root metadata file with the 3 currently valid keys (listed in the previous root file) and include a new key in the place of the oldest key. So, you would sign r1 using <k1,k2,k3> and list <k2,k3,k4> as the root keys in that file. If you had all of the root metadata from your last-known-good to the current one, you could validate the keys yourself.

We were discussing having root files listed on the file system by version number. So if the version of your previous root file was 7 and you just got version 10, you would know to download [89].root.json to find the missing keys and metadata. Note that each of these files only contains signatures from a constant number of keys. If all intermediate root files are downloaded, you can discard old files.

Can you clarify if I've misunderstood anything? Will this change address your concerns?

On a semi-related note, we also are open to adding the hash of the previous piece of metadata to the current version to help with tamper-resistance. If this is desirable, we should start another issue to discuss this topic. There are some complications / corner cases that need to be thought through...

Also, could we please move the "pinning" proposal / discussion to a separate issue?

On Fri, Jul 15, 2016 at 10:39 AM, Evan Cordell notifications@github.com wrote:

This proposes a change to the TUF spec to support a tamper-evident public log of changes to the root authority in a TUF repository and to allow trust-pinning for delegated roles.

There is further discussion of this idea at docker/notary#458 https://github.com/docker/notary/issues/458 and docker/notary#835 https://github.com/docker/notary/issues/835 Motivation

From the TUF spec:

To replace a compromised root key or any other top-level role key, the root role signs a new root.json file that lists the updated trusted keys for the role. When replacing root keys, an application will sign the new root.json file with both the new and old root keys until all clients are known to have obtained the new root.json file (a safe assumption is that this will be a very long time or never).

Currently, rotating the root keys requires signing with old keys indefinitely. Although this does successfully allow rotation, supporting even moderately aggressive rotation schemes (say, having a root threshold of 2/3 available keys and rotating those keys quarterly) results in a large number of old keys that need to be kept around.

Note that this may be less of a concern in TUF as-specified:

Periodically, the software update system using the framework instructs the framework to check each repository for updates.

If clients can be assumed to be constantly polling, there may be an opportunity to remove older keys. But this is not the case for package managers in general - current partial/full implementations in PyPI, RubyGems, and Notary do not have "periodic" polling as one of their properties. Proposal

The canonical (server) TUF metadata repository should version the root.json file any time root keys change, and should point to a "parent" root.json.

{ "signed": { "_type": "Root", // ... "parent": "123" // sha256 of previous root.json }, "parentSignatures" { // current metadata signed with parent keys/threshold }, "signatures": [ // current metadata signed with current keys/threshold ] }

When a client gets new metadata with a parent and parentSignatures block, it does the following:

  • requests the parent metadata file from the server by hash
  • verifies the hash
  • using the public keys and threshold of the parent, calculates the signatures in parentSignatures of the new metadata
  • repeats the process if the parent also has a parent block, until either the hash matches the current local metadata or the pulled metadata doesn't have a parent block.

Example

Rotating a single root key from A to B

  • switch public key in public portion to point to B.pub
  • calculate hash of original metadata and put it in the parent block of the new metadata
  • sign new metadata with B.key and put in signatures block.
  • sign new metadata with A.key and put in parentSignatures block.
  • publish new metadata
  • old metadata should still be accessible by hash (not name)

Pinning Delegated Trust

Trust pinning is unspecified in TUF, though it does exist in Notary, which requires pinning an x509 cert or CA.

Using a public log on delegations would allow trust pinning with key rotation. Instead of pinning to a CA or cert, a pubic key (ideally, a set of public keys) would be pinned.

If trust is pinned for a delegation, we require that there is a valid parent chain back to the pinned public keys. This works analagously to the root key rotation case.

An interesting property that falls out of this is that users can decide where to root their trust (the registry or any delegated target owner) without losing the TUF freshness/rotation ability.

We leave the semantics of specifying pinned trust unspecified for now. Example

Rotating a pinned delegation key

  • trust is pinned for a delegation (e.g. targets/namespace) to namespace.pub
  • the namespace owner generates a new keypair and new namespace metadata
  • the namespace owner calculates the hash of original metadata and puts it in the parent block of the new metadata
  • the namespace owner signs the new metadata with new_namespace.pub and puts that signature in the signatures block
  • the namespace owner signs the new metadata with namespace.pub, the old key, and puts that signature in the parentSignatures block

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/theupdateframework/tuf/issues/340, or mute the thread https://github.com/notifications/unsubscribe-auth/AA0XD1z0cJRwe7HT9Qlwb7IJHtUcnxqWks5qV5uvgaJpZM4JNdo6 .

ecordell commented 8 years ago

Sorry for the delay, must have missed the email about this.

I think the issue you want to solve is efficiently getting a chain of trust from the oldest known to current piece of root metadata.

Correct - and your description is also correctly restating this proposal.

we also are open to adding the hash of the previous piece of metadata to the current version to help with tamper-resistance.

I think this is desirable for two reasons:

But this can be discussed in a separate issue.

To summarize, three things are being discussed:

  1. Rotating keys by signing new metadata with new and previous keys
  2. Adding a hash of parent metadata
  3. Using the same rotation scheme as (1) for delegated metadata to support pinning

I can create separate issues for these for further discussion.

JustinCappos commented 8 years ago

Okay, so on the topic of this issue (rotating keys be signing new metadata), I'd like the suggest the edit previously mentioned design change.

I propose we update the spec to require that root files are accessible on disk using their version number. For example, 1.root.json, 2.root.json, etc. The values used in the file names must be absolute (exactly containing the version number) and not an offset or similar. The rationale is to enable a client to easily retrieve all missing files.

I also propose we include an example of how to do key rotation that includes text similar to what I listed in my previous post. It should include which keys are used in which iteration and how a client rotates out keys while maintaining trust.

So, if anyone has any feedback please let's start to chime in. A PR to the TUF 1.0 spec that makes these changes would also be appreciated.

Thanks, Justin

On Wed, Jul 20, 2016 at 11:59 AM, Evan Cordell notifications@github.com wrote:

Sorry for the delay, must have missed the email about this.

I think the issue you want to solve is efficiently getting a chain of trust from the oldest known to current piece of root metadata.

Correct - and your description is also correctly restating this proposal.

we also are open to adding the hash of the previous piece of metadata to the current version to help with tamper-resistance.

I think this is desirable for two reasons:

  • Requests including the expected hash would signal to the server the client has forked
  • There may be cases where forking would be desirable (instead of being an indication of key compromise) - perhaps this would allow a repo to bless "active" mirrors?

But this can be discussed in a separate issue.

To summarize, three things are being discussed:

  1. Rotating keys by signing new metadata with new and previous keys
  2. Adding a hash of parent metadata
  3. Using the same rotation scheme as (1) for delegated metadata to support pinning

I can create separate issues for these for further discussion.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/theupdateframework/tuf/issues/340#issuecomment-233995392, or mute the thread https://github.com/notifications/unsubscribe-auth/AA0XD9PNu1oTj8jo-5QTBUUKjUPpogcwks5qXkXxgaJpZM4JNdo6 .

awwad commented 6 years ago

I gather that this has been settled by #341? It doesn't provide a log, exactly, but root chaining seems to provide for the listed needs. Closable, @ecordell?

ecordell commented 6 years ago

Yep, #341 was to address this. Closing.