Compliance with metadata specification UVF - Universal Vault Format (Key Rotation)

chenkins commented 1 year ago

Story

Persona:
Need:
Purpose:

Acceptance Criteria

[ ] ~shortening: nothing to do?~
[ ] ~extended attributes or migration plan for extended attributes~
[x] implementation vault JWE = metadata.uvf
[ ] implementation new file header format
[ ] documentation

Open Questions

Context

Implementation

chenkins commented 1 year ago

@overheadhunter could you detail on what we need for v1 with respect to https://iterate-ch.atlassian.net/wiki/spaces/ITERATE/pages/26574860/2023-09-13+Meeting+notes ?

chenkins commented 1 year ago

Ich werden den Key Rotation Mechanismus mal ausdefinieren und dabei Migrationspfade im Blick behalten. xattr können wir problemlos zum bestehenden Format hinzufügen. Max Dateinamens/Pfad-Länge sollten wir hard cappen und auf Shortening verzichten. Damit haben wir ein robustes Fundament

chenkins commented 1 year ago

@overheadhunter what about https://github.com/encryption-alliance/unified-vault-format/pull/19/files - does this imply modifications for v1?

overheadhunter commented 1 year ago

Yes I plan to support key rotation from day 1, as adding it later is a risky migration that may be exploited by downgrade attacks.

chenkins commented 9 months ago

@overheadhunter do we need more than https://github.com/shift7-ch/cipherduck-hub/pull/1/files#r1448542864 for v1?

overheadhunter commented 9 months ago

@overheadhunter do we need more than https://github.com/shift7-ch/cipherduck-hub/pull/1/files#r1448542864 for v1?

This one is unrelated to key rotation. So, this one is not sufficient. I'll try to draft the minimal diff required to add key rotation as specified in https://github.com/encryption-alliance/unified-vault-format/pull/19 to our existing vault format.

chenkins commented 9 months ago

Discussion with @overheadhunter

Migration MUST requirements

no re-encryption of files
resumable

Migrations SHOULD requirements

move/rename operations undesirable

Cipherduck Strategy We do a merge

new metadata format and new file header format
vault8 directory layout

Implementation

metadata format replaces key in Vault Key (JWE), i.e. vault membership revocation triggers key rotation (new JWE for all users).
new encrypted file header, adding reference to file key in file header of encrypted file (statically the one from the metadata "file", no key rotations for now) etc.

Migration risks w.r.t. uvf-to-be

conceptually nothing new regarding encryption
- global file name encryption key (nameFormat/nameKey)
- global file content encryption algorithm (fileFormat)
no shortening (although this will surely not be part of uvf):
- additional complexity (metadata file for the mapping between the shortenendd and full path) for not being reallyrequired (see analysis https://community.cryptomator.org/t/handling-of-long-filenames-in-cryptomator-1-5-0/4191)
- not relevant for S3, S3 has key length restriction of 1024 UTF-8 chars (https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-keys.html)
- not used in Cyberduck anyway
directory layout/flattening: 3 options, we cannot decide before uvf is finalized 1) forced migration - needs blocking access to vaults during migration 2) support multiple versions concurently, only new vaults would support the new vault format, (key rotation for existing vaults independent of uvf) 3) we're independent, eternally our own format/standard
extended attributes: new feature, no migration

Key rotation

selling point: revocation not only at ACL level, but also cryptographically ensured (to new data)
- race conditions: not too many rotations expected (although removing users from a group could come from LDAP, we might to be careful here)
- we might need to have some additional notification mechanisms (push or long polling or....)
- if conflict "new version with new key remote" against "new version with old key local" is detected, triggers polling new metadata.uvf = Vault Key JWE
- other conflict detection/resolution metchanism should otherwise be the same as now for Cyberduck/Mountain Duck

Next steps

check file header implementation in Cyberduck
implement changes to vault JWE format (when vaults are created (hub and client) and when vault is connected to (=same as unlocked see #4) in client): get latest File Key instead of key attribute in JWE.

chenkins commented 9 months ago

As far as I can see, cryptolib is used with

final FileHeader header = cryptomator.getFileHeaderCryptor().create();
cryptomator.getFileHeaderCryptor().encryptHeader(header);

Example

@overheadhunter 1) Would it make sense to extend FileHeaderCryptor.create() to accept the Vault Key ID? 2) File Body Encryption does not change w.r.t to vault format 8, right?

We then need to pass through the Vault Key ID from vault loading to header creation. (To be discussed with @ylangisc / @dkocher ).

chenkins commented 9 months ago

Discussion with @overheadhunter

UVF spec requires symmetric encryption of the metadata (a "alg": "A256KW" based JWE) as it should be possible to put the metadata file to S3 (for use cases other than Cipherduck).
```
1. device key (EC) decrypts user key (EC)
2. user key (EC) decrypts vault masterkey (AES)
3. vault masterkey (AES) decrypts vault key
```
@overheadhunter will open pr in cryptolib with API for new file header (accepting Vault Key ID)
OK to have a 4 char constant for now in the code, no need to pass through (as we do not rotate keys yet).
user key in hub stays the same
```
{ "key": "vault masterkey"}
```

new API returning essentially the vault.uvf-formatted A256KW-based JWE from /api/vaults/{vaultId}/metadata:

{
"fileFormat": "AES-256-GCM-32k",
"nameFormat": "AES-256-SIV",
"keys": {
"QBsJ": "<vault key>",
"HDm3": "<name key>"
},
"latestFileKey": "QBsJ",
"nameKey": "HDm3",
"kdf": "1STEP-HMAC-SHA512",
"com.cipherduck.storage": {
"provider": "references storage profile",
"defaultPath": "bucket name",
"nickname": "display text in client",
"uuid": "vault ID - can be removed",
"region": "bucket region",
"username": "AccessKeyId non-STS",
"password": "SecretKey non-STS"
},
"org.cryptomator.automaticAccessGrant": {
"enabled": true,
"maxWotDepth": -1
}
}

Code

storage
automaticAccessGrant

Terminlogy

a storage profile contains the information which is the same for all vaults of the same flavour (like STS endpoint, AWS roles to be used etc.)
the vault masterkey (aka. VaultKey) is used to decrypt the vault metadata, it is stored as a user-specific JWE under /api/vaults//{vaultId}/access-token
the vault metadata JWE (aka. vault.uvf) contains the vault key(s) (used to encrypt/decrypt the data), it is stored as JWE encrypted by the vault masterkey under /api/vaults/{vaultId}/metadata.

chenkins commented 8 months ago

WiP specification/dependencies:

chenkins commented 8 months ago

WiP diff hub: https://github.com/shift7-ch/cipherduck-hub/compare/605215b1cab5ee4188223bed05336ab4af29a8f6...feature/cipherduck WiP diff client: https://g.iterate.ch/projects/ITERATE/repos/mountainduck/compare/diff?sourceBranch=refs%2Fheads%2Ffeature%2Fcipherduck&targetRepoId=11&targetBranch=2af15e5a2bd7db93a2a37f92a88bab07cadabd65

chenkins commented 8 months ago

Need to discuss with @overheadhunter

recovery key handling with key rotation?
directory/file key hashing
cryptolib plug file key and name key into cryptolib

chenkins commented 8 months ago

pro memoria:

0. setupCode () decrypts user key (EC) --> pbes2

1. device key (EC) decrypts user key (EC) --> ecdhEs
2. user key (EC) decrypts vault masterkey (AES) --> ecdhEs
3. vault masterkey (AES) decrypts vault key --> a256kw

overheadhunter commented 8 months ago

recovery key handling with key rotation?

I think this is impossible in its current implementation. Instead we could add (as already requested a couple of times) recovery contacts. While regular vault members get the key immediately during key rotation, recovery contacts don't need to be members and only get part of the key, requiring 4 eyes (or more) to recover a vault

shift7-ch / katta-server