t2ym
commented
5 years ago Protocol Design (Tentative) in Pseudo-Code
Design Issues
-
[ ] Encrypted heading 4 bytes (6 chars in Base64) of SessionID change regularly according to
session_timestampas the key and the iv for encryption are fixed for each version. Can this be a potential vulnerability? Should this be obfuscated as in TLS 1.3 session lifetime? -
[ ] Current implementation does not have any
"Reject"response for invalid"Connect"and"Update"requests but just responds with a fake"Accept"response starting with0x02(RecordType.Accept) followed by random bytes. Is this feasible? -
[ ] How can selective encryption be implemented?
- For enterprise proxies to detect undesirable traffic while preserving security for the code
- Encrypt all the code (HTML, JavaScript, CSS, etc.)
- Do not encrypt the user data (POST data, GET contents) but their integrity is verified
- Can a custom header such as
x-auditing-proxy: contentsbe inserted at the proxy to request the server and the client to decrypt contents selectively? If so, can an auditing notification message be shown so that the user can understand their contents are open to the auditing proxy?
-
[ ] Validation of
ClientIntegrity.browserHashis going to be handled in a separate component.
-
_dummy_item_forindentation- [ ] Key Concepts
- [ ] Released versions of browsers are Pre-shared Secrets for authentication
- [ ] Evergreen browsers are evolving so fast that their
windowobject structures can be their unique fingerprints - [x] Each version of Chrome, Chrome Beta, Chrome Canary, Chromium can be identified
- [ ] How to handle released Chrome browsers with experimental features enabled?
- Treat them as compromised browsers and block them
- [ ] Accesses from validated browsers by agents/manual validation URLs are stored at
validationService.jsrepository
- [ ] Evergreen browsers are evolving so fast that their
- [ ] Identify browsers with
ClientIntegrity.browserHashassociated withuser-agent- [ ]
browserHashmust be kept secret, unpredictable, but deterministic - [ ]
browserHashvaries on- browser versions
- application versions
integrity.jssource code including RSA, ECDSA public keys in base64- other deterministic entropy such as
new Error().stack - TBD
- [ ] Device/User-specific private configurations are excluded as noises
- [x] Issue: inconsistent browserHash with URL search parameters
- [x] Root Cause:
document.locationis missing in the excluded volatile object list - [x] Fixed at 0.4.0-alpha.49
- Currently excluded properties in
integrity.js/* volatileXXX patterns for excluding device/user-specific data as noises for the browser hash */ // device/user-specific number data to be excluded as noises const volatileNumbers = [ '.document:object.timeline:object.currentTime:number', // current time '.innerWidth:number', // window size '.innerHeight:number', // window size '.screenX:number', // screen size '.screenY:number', // screen size '.outerWidth:number', // window size '.outerHeight:number', // window size '.screenLeft:number', // screen size '.screenTop:number', // screen size '.performance:object.timeOrigin:number', // current time '.visualViewport:object.width:number', // window size '.visualViewport:object.height:number', // window size '.devicePixelRatio:number', // screen device '.navigator:object.connection:object.downlink:number', // network connection '.navigator:object.connection:object.downlinkMax:number', // network connection '.navigator:object.connection:object.rtt:number', // network connection '.navigator:object.maxTouchPoints:number', // touch screen device feature '.navigator:object.hardwareConcurrency:number', // CPU device information '.navigator:object.deviceMemory:number', // Memory device information '.history:object.length:number', // history ]; const volatileNumbersSet = new Set(); volatileNumbers.forEach(pos => volatileNumbersSet.add(pos)); // patterns for device/user-specific number data to be excluded as noises const volatileNumbersRegExp = new RegExp( '^((\.performance:object\.timing:object|' + // current time in performance object '\.performance:object\.memory:object|' + // memory information in performance object '\.console:object\.memory:object|' + // memory information in console API '\.navigator:object\.connection:object|' + // network connection information in navigator object '\.screen:object).*|' + // screen device information '.*(:object\.offsetWidth:number|' + // window/frame size ':object\.scrollWidth:number|' + // window/frame size ':object\.scrollHeight:number|' + // window/frame size ':object\.clientWidth:number|' + // window/frame size ':object\.clientHeight:number))$'); // window/frame size // device/user-specific string data to be excluded as noises const volatileStrings = [ '.navigator:object.connection:object.type:string', // network connection type '.navigator:object.connection:object.effectiveType:string', // network connection type '.navigator:object.language:string', // language preference '.navigator:object.userAgent:string', // user agent, which is sent separately as userAgentHash '.navigator:object.appVersion:string', // app version, which can be detectable from user agent '.navigator:object.platform:string', // platform OS, which can be detectable from user agent '.document:object.referrer:string', // navigation history '.document:object.lastModified:string', // navigation timing '.document:object.URL:string', // document URL '.document:object.documentURI:string', // document URI '.document:object.visibilityState:string', // document visiblitity state '.document:object.webkitVisibilityState:string', // document visiblitity state ]; const volatileStringsSet = new Set(); volatileStrings.forEach(pos => volatileStringsSet.add(pos)); const volatileStringsRegExp = new RegExp( '^.*(\.baseURI:string)$'); // baseURI // device/user-specific object data to be excluded as noises const volatileObjectsRegExp = new RegExp( '^(\.navigator:object\.languages:object|' + // language preferences '\.navigator:object\.serviceWorker:object\.controller:object|' + // Service Worker status '\.performance:object\.navigation:object|' + // navigation history '\.screen:object\.orientation:object|' + // screen device '\.location:object|' + // location object '\.document:object\.location:object|' + // document.location object '\.localStorage:object|' + // browser storage '\.sessionStorage:object).*$'); // browser storage const volatileBooleans = [ '.navigator:object.connection:object.saveData:boolean', // a reduced data usage option '.document:object.hidden:boolean', // hidden status of the document '.document:object.webkitHidden:boolean', // hidden status of the document ];
- [ ]
-
_dummy_item_forindentation- [ ] Validation Protocol
- [ ]
/update- Update in every second, merge atvalidationService.js, and respond replica- [ ] POST data: dictionary of
UA:hash->count, etc. since the last/update - [ ] POST response: dictionary of
UA:hash->status,count, etc.
- [ ] POST data: dictionary of
- [ ] Repository at
validationService.jsServer- [ ]
UA:hex(hash)->status,count, etc.
- [ ]
- [ ] Repository Replica at
validationService.jsclient API forintegrityService.js- [ ]
UA:hex(hash)->status,count, etc. - [ ] Log: Dictionary of
UA:hex(hash)->count, etc. since the last/update
- [ ]
- [ ] Validation at
validationService.jsclient API forintegrityService.js- [ ]
RepoReplica[UA:hex(hash)].status === "validated" - [ ] Increment
Log[UA:hex(hash)].count, etc.
- [ ]
- [ ] Validation URL
- [ ] (tentative)
entryURL/?validate=base64URL(AES_GCM(action=validate×tamp=X...))
- [ ] (tentative)
- [ ] Version upgrade processes
- [ ] TBD
- [ ] Components
- [ ]
validationService.js- [ ] Skeleton
- [x] Client certificate authentication
- [x] Dummy
/updateAPI
- [ ] Agent
validationAgent.js- Key Concepts
- Automatically launch target browsers for validation like Selenium grid
- Automatically control latest and 2nd latest browser versions
Automation of snapshotting VMs andstopping browser auto-update services might be tricky
- Note:
browserHashchanges viapuppeteermanipulation - [ ] Platforms
- [ ] Windows (via Node.js on WSL or git bash?)
- [ ] macOS
- [ ] Linux
- [ ] Chrome OS?
- [ ] Access via Validation Console and open a validation URL in a new tab?
- [ ] Features
- [ ] Launch validation URL
- [x] PoC Launch and terminate
chrome.exewith a target URL from Node.js
- [x] PoC Launch and terminate
- [ ] Control Chrome Update Service
- [ ] Deploy
n + 1agents to control the lastn(presumably 2) release versions of Chrome browsers - [ ] If
nis 2, 3 agents (Agent0,Agent1,Agent2) will rotate their roles on each updateAgent0runs the 2nd latest Chrome; Update Service is suspendedAgent1runs the latest Chrome; Update Service is suspendedAgent2runs the latest Chrome; Update Service is running- When Chrome on
Agent2is updated, Update Service is suspended onAgent2- Update Service on
Agent0is resumed and Chrome onAgent0is updated to the latest version (soon), which is the same version as that onAgent2 - The 3rd latest Chrome might be invalidated later at an appropriate timing
- Status after the Chrome update until the next update
Agent0runs the latest Chrome; Update Service is runningAgent1runs the 2nd latest Chrome; Update Service is suspendedAgent2runs the latest Chrome; Update Service is suspended
- Update Service on
- [ ] On Windows
- Tasks
schtasks.exe /CHANGE /DISABLE /TN GoogleUpdateTaskMachineCoreschtasks.exe /CHANGE /DISABLE /TN GoogleUpdateTaskMachineUAschtasks.exe /CHANGE /ENABLE /TN GoogleUpdateTaskMachineCoreschtasks.exe /CHANGE /ENABLE /TN GoogleUpdateTaskMachineUA- Services
- Manual startup for "Google Update Service (gupdate)" service
- [ ] Deploy
- [ ] Validation Console
- [ ] Skeleton
- [x] Scaffolded via
npm init @open-wc - [x] Accessible via
validationService.jswith client certificate authentication - [ ] Browser List
- [ ] In design
- [ ] Manual Validation
- [ ] In design
- [ ] Configuration
- [ ] In design
-
[x] Fetching of
cache-bundle.jsonshould be skipped when that for the same version has been loaded- The current implementation unregisters the existing Service Worker before sending a Connect request to avoid too complicated situations with Connect requests proxied via the Service Worker
-
[ ] Multiple tabs try to upgrade the app and send
Connectrequests at once, which should be avoided and serialized with a singleConnectrequest- [ ] How to serialize the connections?
-
[ ] Dynamic HTML responses for
iframeelements are locally cached in spite ofcache-control: no-cacheresponse headers for all encrypted responses including non-HTML responses.- Edge servers like
nginxcan avoid unexpected caching by the headers cache-control: privatedoes not work as expected innginx- Workaround # 1:
Object.defineProperty(cache, 'put', { value: () => Promise.resolve() })to disablecache.put()for dynamic HTML responses with targeted URLs incheckResponse()atcache-bundle.js - [ ] What is a desirable mechanism to control local caching of dynamic HTML responses?
- SPA with a fixed entry page is preferred in
thin-hook. Dynamic HTML responses are not very friendly to the architecture.
- Edge servers like
-
[ ] Leaking of secrets from process memory dumps has to be avoided
- Notes:
- Chromium browsers with debug symbols must not be treated as validated browsers
- If Microsoft would provide official debug symbols for Chromium-based Edge browsers, there would be significant risks of leaking secret key data from memory dumps
- There are no perfect solutions to eliminate raw secret key data from memory dumps completely
- [x] Mitigate risks of leaking
ClientIntegritybyclearingrandomizing and deletingArrayBufferobjects after they are used - Note: It is relatively easy to extract
ClientIntegritydata objects from a memory dump compared with other secrets - [ ] Other secrets
- [ ] Are
wrapKey()andunwrapKey()feasible?- [ ] Wrapping keys are still stored in
ArrayBufferobjects - [ ] Are
CryptoKeyobjects stored with any encryption? - [ ] Wrapped keys must be unwrapped (decrypted) anyway for use, which might impact performance as well
- [ ] Wrapping keys are still stored in
- [ ] TBD
Change Log
- 2019-08-27: Remove
#hashfromx-pathheader according to HTTP/1.1, HTTP/2 RFCs - 2019-09-04: Skip fetching and loading
cache-bundle.jsonwhen it has already been loaded
Pseudo-Code
const RecordType = {
Connect: 1, // client -> server
Accept: 2, // server -> client
Update: 3, // client -> server
};
const key_length = 32; // bytes
const iv_length = 12; // bytes
const salt_length = 32; // bytes
Hash.length = 32; // bytes
Hash = SHA256;
const entryPageURL = "https://host.name/entry/page/"; // must be a directory path
const integrityServiceURL = new URL("integrity", entryPageURL).href; // https://host.name/entry/page/integrity
// at server
keys = {
"version": "version_638",
"rsa-private-key.pem": "-----BEGIN RSA PRIVATE KEY-----\nMIIEowIBAAKCAQEAk4Rm72FO94F4KWny7JRE8YUgAzy0hrjsg3XTnieteK0VGDID\nJUyndaCA7HvmLbMRLLkTLg4hcQxAet8SAIM/iSbo51JtNu3qkDXADON2KnZf5P1Y\n5THL3jkwFQQjucbytF92C5yqrKL5wBSH2VFnnv/WR2Mk/GE8B3dPD6X1TdQWRQpD\n0wuWzx/R4DCg1zXcwjD2KK65VDZ4BB9dk3HX0SrtAiFhgP+kydfwgkhQDhBO3rIb\n+qAzRXLB2a3KkYKO5DVFaNWt0J0qRkR+ShTsYfhHCqRemO4ZXfd2aV0lIEtJwmdO\nIl19ijVv+XjDeXEpDDHMwcp5pbnNb6YKTnRu9wIDAQABAoIBAAqY5Fwl/WpCXsN6\n3Pyp2hoPmjEhV0amWjdHa6Bc8VVN+cn3LcqsKwuEMD7M18hIqN8xnHMeiMB6RNeO\n1tg6lYHgzbJwdXAQv10Ev3sti/uY7WKh4JT2ctLQAOhBl99sr1rN0MkcxBYKzy5B\nS1ENTAhcEKSoNqv6wDk5FPDm1yx0B3qZhrws6EkRt29yOO7bhqjF4c5GfeuAkuDB\n44OJ6B+klxT4YsUbfSkUfnlTw3IBfM37dlBEbLH7f3dMeK0UVq2L1MRBjlmOzk8c\nriuhAxeBNGGcTNdQo2fzTMGesg/Ixqoc0Odh1BB9HVVkoCiNFTnACY2oXh3rotxs\nL2ZRMaECgYEA7VFczuTVxtpzPUmAKrNGk9WRLuW+eb08W4DYOf5yYLhxLMHishnR\nBO+t4pkuoJ8SgH4jvb/mxEBdV2wjlJs2QGYvX9EYTwdezHzcpwK03XFHLwpKpRD1\nsTw6UnucWb+63ZB0r5NyVkHtaxhtpPU6hAlTKZXHocPKahEypnsSKQsCgYEAnyFM\nYNPpcat8DM3YPmau0l6Q2RaNRjlQw+rEZ8zRlzPf4hiwPENjklYM7u0ceIbdrjc3\nWeThLyktsnyNCijrv7VZgmgyrgFL29VsVrfeR3ZwgHeBLUNgcTwggBmANrA7rmBN\nzmXhLa5xdn8PmjYQDNK7qEy/3WnHU0VFuWKvfUUCgYBVqytfnIf3cuBq3V+hCnqN\n32i7j0AFXmSte4OS2+GaPLrON2eId31W1Nbml/mXDhV1wRNR6jZ53epUJrtpZ+Zb\ntQehBTBLRxPXqbNVrspvrfbOal6r28V1p5I+OFUmqOniFcWppAaAUOhN4tGh3My0\n4VDeEC2ynaUySOcJ5h+WJQKBgA3lX4EZIEqf2f5YP2j7mIqgXW/Hq2CVgrsJFkum\nNCtLCWL6GvG4RMqznv+CTzkrNdKP2dKMzSlMJERw4fQgLK4aDQ35QWu2i0RQN9y+\nw7dj3WEqjmpAdvyMbp4hG/QqoZuRp1m9xdMyZ5AcemVSEUa9ZEvHH/4azaA07WjJ\n+F8tAoGBAJUQs3F9GTm+TU6Ggeq1t03vkma76Mi2SPpFdmi56SIjGUEsz/SgxvE9\n/CeWuBPxj+viJ9Yinx3wLXEYGFUfDz8/XTUsu+h8q6HLNtR9O2WNadme4AEiSli7\njiQ21H6pWu0NM5e1rNoPTOCh7RFmignOchDFFpl0vXSNUPLn6zA7\n-----END RSA PRIVATE KEY-----\n",
"rsa-public-key.pem": "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAk4Rm72FO94F4KWny7JRE\n8YUgAzy0hrjsg3XTnieteK0VGDIDJUyndaCA7HvmLbMRLLkTLg4hcQxAet8SAIM/\niSbo51JtNu3qkDXADON2KnZf5P1Y5THL3jkwFQQjucbytF92C5yqrKL5wBSH2VFn\nnv/WR2Mk/GE8B3dPD6X1TdQWRQpD0wuWzx/R4DCg1zXcwjD2KK65VDZ4BB9dk3HX\n0SrtAiFhgP+kydfwgkhQDhBO3rIb+qAzRXLB2a3KkYKO5DVFaNWt0J0qRkR+ShTs\nYfhHCqRemO4ZXfd2aV0lIEtJwmdOIl19ijVv+XjDeXEpDDHMwcp5pbnNb6YKTnRu\n9wIDAQAB\n-----END PUBLIC KEY-----\n",
"ecdsa-private-key.pem": "...",
"ecdsa-public-key.pem": "...",
"session-id-aes-key": "1b8c0dde7832e78bf9097421179d004af5ed83e34ea0c5c067cafa469a3d2b49", // no forward secrecy
"session-id-aes-iv": "79706eec5a6c9bf41bf02d1b", // no forward secrecy
};
ECDSA.integrityJSONSignature = ECDSA.sign(ECDSA.serverPrivateKey, SHA_256.digest(integrityJSON));
// at client (main document; encoded)
RSA.serverPublicKey = spki("rsa-public-key.pem");
ECDSA.serverPublicKey = spki("ecdsa-public-key.pem");
Sessions = [];
// on Connect
CurrentSession = { isConnect: true, session_timestamp: Date.now() };
Sessions.push(CurrentSession); // store Connect Session
//requestId = 0; // TODO: revisit later
AES_GCM.{ clientOneTimeKey, clientOneTimeIv } = { Random(hashSize), Random(ivSize) };
NextSession.clientRandom = Random(hashSize);
NextSession.ECDHE.{ clientPublicKey, clientPrivateKey } = ECDH.generateKeys('prime256v1');
CurrentSession.ClientIntegrity =
userAgentHash (= SHA256.digest(navigator.userAgent)) +
browserHash (= SHA256.digest(JSON.stringify(_traverse(wrapper, window)))) + // TODO: Forward secrecy. timestamp, rotation, etc.
scriptsHash (= SHA256.digest(document.querySelectorAll('script').join('\0'))) +
htmlHash (= SHA256.digest(document.querySelector('html').outerHTML))
}
CurrentSession.connect_early_secret =
HKDF-Expand(0, AES_GCM.clientOneTimeKey + AES_GCM.clientOneTimeIv + NextSession.clientRandom + NextSession.ECDHE.clientPublicKey + CurrentSession.ClientIntegrity);
CurrentSession.connect_salt = HKDF-Expand-Label(connect_early_secret, "salt", "", salt_length)
// request
// initial connection
req.headers = {
"x-method": "POST",
"x-scheme": integrityServiceURL.protocol.replace(/:$/, ''),
"x-authority": integrityServiceURL.host,
"x-path": integrityServiceURL.pathname + integrityServiceURL.search, // Note: integrityServiceURL.hash is removed
"content-type": "application/octet-stream",
//"x-request-id": requestId, // TODO: revisit later
"x-timestamp": Date.now(),
"x-digest": "sha256-" + base64(SHA256.digest(req.body)),
"x-integrity": "x-method,x-scheme,x-authority,x-path,header-list,x-timestamp,x-digest;" +
"hmac-sha256-" + base64(HMAC_SHA256(CurrentSession.connect_salt, headers.join("\n") + "\n")) // TODO: browserHash is inappropriate as it is shared among client browsers with the same version
}
req.body =
RecordType.Connect [1] +
RSA_OAEP.encrypt(RSA.serverPublicKey,
AES_GCM.clientOneTimeKey [32] +
AES_GCM.clientOneTimeIv [12]) [256] +
AES_GCM.encrypt(aesKey = AES_GCM.clientOneTimeKey, iv = AES_GCM.clientOneTimeIv,
NextSession.clientRandom [32] +
NextSession.ECDHE.clientPublicKey [65] +
CurrentSession.ClientIntegrity (= userAgentHash + browserHash + scriptsHash + htmlHash) [128] +
)
// session update (Service Worker)
req.headers = {
"x-method": "POST",
"x-scheme": integrityServiceURL.protocol.replace(/:$/, ''),
"x-authority": integrityServiceURL.host,
"x-path": integrityServiceURL.pathname + integrityServiceURL.search, // Note: integrityServiceURL.hash is removed
"content-type": "application/octet-stream",
"x-session-id": base64(CurrentSession.SessionID),
//"x-request-id": requestId, // TODO: revisit later
"x-timestamp": Date.now(),
"x-digest": "sha256-" + base64(SHA256.digest(req.body)),
"x-integrity": "x-method,x-scheme,x-authority,x-path,header-list,x-session-id,x-timestamp,x-digest;" +
"hmac-sha256-" + base64(HMAC_SHA256(CurrentSession.client_write_salt, headers.join("\n") + "\n"))
}
req.body =
RecordType.Update [1] +
AES_GCM.encrypt(aesKey = CurrentSession.client_write_key, iv = CurrentSession.client_write_iv,
AES_GCM.clientOneTimeKey [32] +
AES_GCM.clientOneTimeIv [12] +
NextSession.clientRandom [32] +
NextSession.ECDHE.clientPublicKey [65]
)
// proxied request (Service Worker)
req.headers = {
"x-method": req.method,
"x-scheme": req.url.protocol.replace(/:$/, ''),
"x-authority": req.url.host,
"x-path": req.url.pathname + req.url.search, // Note: req.url.hash is removed
"x-session-id": base64(CurrentSession.SessionID),
//"x-request-id": requestId, // TODO: revisit later
"x-timestamp": Date.now(),
"x-digest": "sha256-" + base64(SHA256.digest(req.body)), // on POST/PUT
"x-content-encoding": "aes-256-gcm", // on POST/PUT; no gzip support for request body encryption
"x-integrity": "x-method,x-scheme,x-authority,x-path,header-list,x-session-id,x-timestamp,x-digest,x-content-encoding;" +
"hmac-sha256-" + base64(HMAC_SHA256(CurrentSession.client_write_salt, headers.join("\n") + "\n"))
}
req.body =
AES_GCM.encrypt(aesKey = CurrentSession.client_write_key, iv = CurrentSession.client_write_iv,
Body = event.request.body // Note: Body is authenticated by x-digest and x-integrity headers
)
// response
res.headers = {
"x-scheme": req.url.protocol.replace(/:$/, ''),
"x-authority": req.url.host,
"x-path": req.url.pathname + req.url.search, // Note: req.url.hash is removed
//"x-request-id": requestId, // TODO: revisit later
"x-request-timestamp": req.headers["x-timestamp"],
"x-timestamp": Date.now(),
"x-digest": "sha256-" + base64(SHA256.digest(res.body)),
"x-content-encoding": "aes-256-gcm"/"gzip+aes-256-gcm"
"x-integrity": "x-scheme,x-authority,x-path,header-list,x-request-timestamp,x-timestamp,x-digest;" +
"hmac-sha256-" + base64(HMAC_SHA256(CurrentSession.server_write_salt, headers.join("\n") + "\n"))
}
res.body =
AES_GCM.encrypt(CurrentSession.server_write_key, CurrentSession.server_write_iv,
// no explicit field for body length
Body = identity/gzip(response.body)
)
// at server
encrypted = req.body;
CurrentSession = {};
// Validate headers
req.headers['content-type'] === 'application/octet-stream';
req.headers['x-method'] === req.method === 'POST';
req.headers['x-scheme'] === req.url.protocol.replace(/:$/, ''); // TODO: validate req.url
req.headers['x-authority'] === req.url.host;
req.headers['x-path'] === req.url.pathname + req.url.search; // Note: req.url.hash is removed
req.headers['x-timestamp'] within acceptable_timestamp_range // validate clock of the client
req.headers['x-digest'] === 'sha256-' + base64(SHA256.digest(req.body));
if (RecordType.type === RecordType.Connect) {
// Validate Connect
// decrypt Connect
RecordType.type = encrypted.subarray(0, 1) === RecordType.Connect
[ AES_GCM.clientOneTimeKey, AES_GCM.clientOneTimeIv ] = RSA_OAEP.decrypt(RSA.serverPrivateKey, encrypted.subarray(1, 1 + keySize));
[ NextSession.clientRandom, NextSession.ECDHE.clientPublicKey, CurrentSession.ClientIntegrity ] =
AES_GCM.decrypt(AES_GCM.clientOneTimeKey, AES_GCM.clientOneTimeIv, encrypted.subarray(1 + keySize, encrypted.byteLength));
// save at build time; verify at runtime
SHA_256.digest(req.header['user-agent']) === userAgentHash;
keys["scriptsHashHex", "htmlHashHex"] (=:build, ===:runtime) [toHex(scriptsHash), toHex(htmlHash)];
// Validate browserHash at integrityService
browserHash === verifiedBrowserHash(req.header['user-agent']) // Note: This is the key to the authentication in the handshake
// Derive connect_early_secret
CurrentSession.connect_early_secret =
HKDF-Extract(0, AES_GCM.clientOneTimeKey + AES_GCM.clientOneTimeIv + NextSession.clientRandom + NextSession.ECDHE.clientPublicKey + CurrentSession.ClientIntegrity);
// Derive Pseudo-PSK for initial key derivation
CurrentSession.PSK = HKDF-Expand-Label(CurrentSession.connect_early_secret, "connect", "", Hash.length) // pseudo-PSK
}
else if (RecordType.type === RecordType.Update && req.header['x-session-id']) {
// decrypt SessionID
CurrentSession.SessionID = Buffer.from(req.header['x-session-id'], 'base64');
CurrentSession.SessionIDPayload = AES_GCM.decrypt(sessionIdAESKey, sessionIdAESIv, SessionID);
CurrentSession.[ session_timestamp [4], master_secret [32], transcript_hash = Transcript-Hash(Connect/Update + Accept.header) [32] ] = CurrentSession.SessionIDPayload;
// Validate session_timestamp
CurrentSession.session_timestamp within expected lifetime
// Derive current secrets (not for the next updated ones)
CurrentSession.client_traffic_secret = HKDF-Expand-Label(CurrentSession.master_secret, "c ap traffic", CurrentSession.transcript_hash, Hash.length)
//CurrentSession.server_traffic_secret = HKDF-Expand-Label(CurrentSession.master_secret, "s ap traffic", CurrentSession.transcript_hash, Hash.length)
CurrentSession.session_master_secret = HKDF-Expand-Label(CurrentSession.master_secret, "session", CurrentSession.transcript_hash, Hash.length)
//CurrentSession.server_write_key = HKDF-Expand-Label(CurrentSession.server_traffic_secret, "key", "", key_length);
//CurrentSession.server_write_iv = HKDF-Expand-Label(CurrentSession.server_traffic_secret, "iv", "", iv_length);
//CurrentSession.server_write_salt = HKDF-Expand-Label(CurrentSession.server_traffic_secret, "salt", "", salt_length);
CurrentSession.client_write_key = HKDF-Expand-Label(CurrentSession.client_traffic_secret, "key", "", key_length);
CurrentSession.client_write_iv = HKDF-Expand-Label(CurrentSession.client_traffic_secret, "iv", "", iv_length);
CurrentSession.client_write_salt = HKDF-Expand-Label(CurrentSession.client_traffic_secret, "salt", "", salt_length);
CurrentSession.PSK = HKDF-Expand-Label(CurrentSession.session_master_secret, "update", "", Hash.length)
// decrypt Update
RecordType.type = encrypted.subarray(0, 1) === RecordType.Update
[ AES_GCM.clientOneTimeKey, AES_GCM.clientOneTimeIv, NextSession.clientRandom, NextSession.ECDHE.clientPublicKey ] =
AES_GCM.decrypt(CurrentSession.client_write_key, CurrentSession.client_write_iv, encrypted.subarray(1, encrypted.byteLength));
}
// Validate x-integrity (delayed)
// parse headers
[ headerNamesCSV, 'hmac-sha256-' + headerSignatureBase64 ] = req.headers['x-integrity'].split(';')
headerNames = headersCSV.split(',');
headerSignature = atob(headerSignatureBase64);
headers = headerNames.map((headerName) => headerName + ': ' + req.headers[headerName] + '\n').join('');
// for Connect
connect_salt = HKDF-Expand-Label(CurrentSession.connect_early_secret, "salt", "", salt_length)
salt = connect_salt;
// for Update
salt = CurrentSession.client_write_salt;
// verify hmac
HMAC_SHA256(salt, headers) === headerSignature;
// Request has been validated
// prepare the next session
NextSession = {};
NextSession.serverRandom = Random(hashSize);
NextSession.ECDHE.{ serverPublicKey, serverPrivateKey } = ECDH.generateKeys('prime256v1');
NextSession.ECDHE.sharedKey = ECDH.deriveKey(NextSession.ECDHE.serverPrivateKey, NextSession.ECDHE.clientPublicKey);
RFC5869
HKDF-Extract(salt, IKM) = HMAC-Hash(salt, IKM)
Inputs:
salt optional salt value (a non-secret random value); if not provided, it is set to a string of HashLen zeros.
IKM input keying material
HKDF-Expand(PRK, info, L) -> OKM
Options:
Hash a hash function; HashLen denotes the length of the hash function output in octets
Inputs:
PRK a pseudorandom key of at least HashLen octets (usually, the output from the extract step)
info optional context and application specific information (can be a zero-length string)
L length of output keying material in octets (<= 255*HashLen)
Output:
OKM output keying material (of L octets)
The output OKM is calculated as follows:
N = ceil(L/HashLen)
T = T(1) | T(2) | T(3) | ... | T(N)
OKM = first L octets of T
where:
T(0) = empty string (zero length)
T(1) = HMAC-Hash(PRK, T(0) | info | 0x01)
T(2) = HMAC-Hash(PRK, T(1) | info | 0x02)
T(3) = HMAC-Hash(PRK, T(2) | info | 0x03)
...
RFC8446
Transcript-Hash(M1, M2, ... Mn) = Hash(M1 || M2 || ... || Mn)
HKDF-Expand-Label(Secret, Label, Context, Length) =
HKDF-Expand(Secret, HkdfLabel, Length)
Where HkdfLabel is specified as:
struct {
uint16 length = Length;
opaque label<7..255> = "tls13 " + Label;
opaque context<0..255> = Context;
} HkdfLabel;
Derive-Secret(Secret, Label, Messages) =
HKDF-Expand-Label(Secret, Label,
Transcript-Hash(Messages), Hash.length)
Customized Key Schedule:
0
|
v
PSK -> HKDF-Extract = Early Secret
|
v
Derive-Secret(., "derived", "")
|
v
ECDHE -> HKDF-Extract = Handshake Secret
|
v
Derive-Secret(., "derived", "")
|
v
0 -> HKDF-Extract = Master Secret
|
+-----> Derive-Secret(., "c ap traffic",
| Connect/Update + Accept.header)
| = client_traffic_secret
|
+-----> Derive-Secret(., "s ap traffic",
| Connect/Update + Accept.header)
| = server_traffic_secret
|
+-----> Derive-Secret(., "session",
Connect/Update + Accept.header)
= session_master_secret
[sender]_write_key = HKDF-Expand-Label(Secret, "key", "", key_length)
[sender]_write_iv = HKDF-Expand-Label(Secret, "iv", "", iv_length)
[sender]_write_salt = HKDF-Expand-Label(Secret, "salt", "", salt_length)
Accept.header =
RecordType.Accept [1] +
AES_GCM.encrypt(AES_GCM.clientOneTimeKey, AES_GCM.clientOneTimeIv,
NextSession.serverRandom [32] +
NextSession.ECDHE.serverPublicKey [32]) [80]
// Derive next secrets
NextSession.early_secret = HKDF-Extract(0, CurrentSession.PSK);
NextSession.handshake_secret = HKDF-Extract(Derive-Secret(NextSession.early_secret, "derived", ""), NextSession.ECDHE.sharedKey);
NextSession.master_secret = HKDF-Extract(Derive-Secret(NextSession.handshake_secret, "derived", ""), 0);
NextSession.transcript_hash = Transcript-Hash(Connect/Update + Accept.header);
//NextSession.client_traffic_secret = HKDF-Expand-Label(NextSession.master_secret, "c ap traffic", NextSession.transcript_hash, Hash.length)
NextSession.server_traffic_secret = HKDF-Expand-Label(NextSession.master_secret, "s ap traffic", NextSession.transcript_hash, Hash.length)
//NextSession.session_master_secret = HKDF-Expand-Label(NextSession.master_secret, "session", NextSession.transcript_hash, Hash.length)
NextSession.server_write_key = HKDF-Expand-Label(NextSession.server_traffic_secret, "key", "", key_length);
NextSession.server_write_iv = HKDF-Expand-Label(NextSession.server_traffic_secret, "iv", "", iv_length);
NextSession.server_write_salt = HKDF-Expand-Label(NextSession.server_traffic_secret, "salt", "", salt_length);
//NextSession.client_write_key = HKDF-Expand-Label(NextSession.client_traffic_secret, "key", "", key_length);
//NextSession.client_write_iv = HKDF-Expand-Label(NextSession.client_traffic_secret, "iv", "", iv_length);
//NextSession.client_write_salt = HKDF-Expand-Label(NextSession.client_traffic_secret, "salt", "", salt_length);
NextSession.session_timestamp_raw = Date.now();
NextSession.session_timestamp = htonl(Uint32Array.of(Math.floor(NextSession.session_timestamp_raw / 1000)));
SessionIDPayload =
NextSession.session_timestamp [4] +
NextSession.master_secret [32] +
NextSession.transcript_hash [32];
NextSession.SessionID = AES_GCM.encrypt(sessionIdAESKey, sessionIdAesIv,
SessionIDPayload) [84];
Accept.body =
AES_GCM.encrypt(NextSession.server_write_key, NextSession.server_write_iv,
NextSession.SessionID [84] +
ECDSA.integrityJSONSignature [64]
) [160]
Accept [241] = Accept.header [81] + Accept.body [160]
// response
res.headers = {
"x-status": res.status,
"x-scheme": req.url.protocol.replace(/:$/, ''),
"x-authority": req.url.host,
"x-path": req.url.pathname + req.url.search, // Note: req.url.hash is removed
"content-type": "application/octet-stream",
//"x-request-id": requestId, // TODO: revisit later
"x-request-timestamp": req.headers["x-timestamp"],
"x-timestamp": Date.now(),
"x-digest": "sha256-" + base64(SHA256.digest(res.body)),
"x-integrity": "x-status,x-scheme,x-authority,x-path,header-list,x-request-timestamp,x-timestamp,x-digest;" +
"hmac-sha256-" + base64(HMAC_SHA256(NextSession.server_write_salt, headers.join("\n") + "\n"))
}
res.body = Accept
// at client (main document; encoded)
session_early_lifetime = 300 * 1000 (msec)
session_lifetime = 600 * 1000 (msec)
NextSession = { session_timestamp_raw: Date.now() }; // TODO: number or htonl(Date.now()/1000)?
// Validate headers
res.headers['x-status'] === res.status;
res.headers['content-type'] === 'application/octet-stream';
res.headers['x-scheme'] === req.url.protocol.replace(/:$/, ''); // TODO: validate req.url
res.headers['x-authority'] === req.url.host;
res.headers['x-path'] === req.url.pathname + req.url.search; // Note: req.url.hash is removed
res.headers['x-request-timestamp'] === req.headers['x-timestamp'];
res.headers['x-timestamp'] within acceptable_timestamp_range // check timestamp with estimated clock difference
res.headers['x-digest'] === 'sha256-' + base64(SHA256.digest(res.body));
// decrypt res.body
// decrypt Accept.header
[ NextSession.serverRandom, NextSession.ECDHE.serverPublicKey ] =
AES_GCM.decrypt(aesKey = AES_GCM.clientOneTimeKey, iv = AES_GCM.clientOneTimeIv, res.body.subarray(1, Accept.header.byteLength));
// Derive Secrets
NextSession.ECDHE.sharedKey = ECDH.deriveKey(NextSession.ECDHE.clientPrivateKey, NextSession.ECDHE.serverPublicKey);
// For Connect
CurrentSession.PSK = HKDF-Expand-Label(CurrentSession.connect_early_secret, "connect", "", Hash.length) // pseudo-PSK
// For Update
CurrentSession.PSK = HKDF-Expand-Label(CurrentSession.session_master_secret, "update", "", Hash.length)
NextSession.early_secret = HKDF-Extract(0, CurrentSession.PSK);
NextSession.handshake_secret = HKDF-Extract(Derive-Secret(NextSession.early_secret, "derived", ""), NextSession.ECDHE.sharedKey);
NextSession.master_secret = HKDF-Extract(Derive-Secret(NextSession.handshake_secret, "derived", ""), 0);
NextSession.transcript_hash = Transcript-Hash(Connect/Update + Accept.header);
NextSession.client_traffic_secret = HKDF-Expand-Label(NextSession.master_secret, "c ap traffic", NextSession.transcript_hash, Hash.length)
NextSession.server_traffic_secret = HKDF-Expand-Label(NextSession.master_secret, "s ap traffic", NextSession.transcript_hash, Hash.length)
NextSession.session_master_secret = HKDF-Expand-Label(NextSession.master_secret, "session", NextSession.transcript_hash, Hash.length)
NextSession.server_write_key = HKDF-Expand-Label(NextSession.server_traffic_secret, "key", "", key_length);
NextSession.server_write_iv = HKDF-Expand-Label(NextSession.server_traffic_secret, "iv", "", iv_length);
NextSession.server_write_salt = HKDF-Expand-Label(NextSession.server_traffic_secret, "salt", "", salt_length);
NextSession.client_write_key = HKDF-Expand-Label(NextSession.client_traffic_secret, "key", "", key_length);
NextSession.client_write_iv = HKDF-Expand-Label(NextSession.client_traffic_secret, "iv", "", iv_length);
NextSession.client_write_salt = HKDF-Expand-Label(NextSession.client_traffic_secret, "salt", "", salt_length);
// decrypt Accept.body
[ NextSession.SessionID, ECDSA.integrityJSONSignature ] =
AES_GCM.decrypt(aesKey = NextSession.server_write_key, iv = NextSession.server_write_iv, res.body.subarray(Accept.header.byteLength));
// Validate x-integrity (delayed)
// parse headers
[ headerNamesCSV, 'hmac-sha256-' + headerSignatureBase64 ] = res.headers['x-integrity'].split(';')
headerNames = headerNamesCSV.split(',');
headerSignature = atob(headerSignatureBase64);
headers = headerNames.map((headerName) => headerName + ': ' + req.headers[headerName] + '\n').join('') + '\n';
// verify hmac
HMAC_SHA256(NextSession.server_write_salt, headers) === headerSignature;
// Register NextSession (TODO: at this timing?)
Sessions.push(NextSession);
CurrentSession = NextSession; // update current session with the next session
// On Connect
// verify integrityJSON signature
integrityJSON = decrypt(fetch('integrity.json', {
headers: {
"x-method": "GET",
"x-scheme": "https",
"x-authority": req.url.host,
"x-path": entryPageURL.pathname + "integrity.json",
"x-session-id": base64(CurrentSession.SessionID),
//"x-request-id": requestId, // TODO: revisit later
"x-timestamp": Date.now(),
"x-integrity": "x-method,x-scheme,x-authority,x-path,header-list,x-session-id,x-timestamp;" +
"hmac-sha256-" + base64(HMAC_SHA256(CurrentSession.client_write_salt, headers.join("\n") + "\n"))
}
}));
ECDSA.verify(ECDSA.serverPublicKey, ECDSA.integrityJSONSignature, integrityJSON) === true;
// register Service Worker
navigator.serviceWorker.register('hook.min.js');
// verify Service Worker script
fetch('hook.min.js', { integrity: hookMinJsScript.integrity }).ok
// store integrityJSON
caches.open(version).put(INTEGRITY_PSEUDO_URL, integrityJSON);
if (caches.open(version).match(CACHE_STATUS_PSEUDO_URL)) {
// skip fetching and loading cache-bundle.json
}
else {
// load cache bundle
cacheBundle = decrypt(fetch('cache-bundle.json', {
headers: {
"x-method": "GET",
"x-scheme": "https",
"x-authority": req.url.host,
"x-path": entryPageURL.pathname + "cache-bundle.json",
"x-session-id": base64(CurrentSession.SessionID),
//"x-request-id": requestId, // TODO: revisit later
"x-timestamp": Date.now(),
"x-integrity": "x-method,x-scheme,x-authority,x-path,header-list,x-session-id,x-timestamp;" +
"hmac-sha256-" + base64(HMAC_SHA256(CurrentSession.client_write_salt, headers.join("\n") + "\n"))
}
}));
'sha256-' + Base64(SHA_256.digest(cacheBundle)) === integrityJSON['cache-bundle.json']; // verify
caches.open(version).put(cacheBundle.entries);
}
// store Sessions
postMessage(['plugin', 'Integrity:enqueue', Sessions]); // transfer session information to the Service Worker
// at Service Worker before loading plugins
hook.parameters.messageQueues['Integrity:enqueue'] = [ event ]; // enqueue the message event with Sessions
// at client
// reload the entry page
// at Service Worker in processing the entry page
data = hook.parameters.messageQueues['Integrity:enqueue'][0].data;
Sessions = data[2];
// at client (main document; decoded)
postMessage(['plugin', 'Integrity:enqueue'], [port]); // request Sessions object from the Service Worker
Sessions = data[2];
// At Service Worker
setTimeout(() => {
for (let session of Sessions) {
if (session.session_timestamp + session_lifetime < Date.now()) {
// Discard invalidated Sessions;
}
}
if (Sessions[Sessions.length - 1].session_timestamp + session_early_lifetime < Date.now()) {
Update Sessions;
}
}, sessionCheckInterval)
Prototyping optional double encryption for integrity
Design Principles
ECDHE-ECDSA-AES256-GCM-SHA384, most of the payloads in ASN.1 become fixed values and can be omitted on handshakingCurrent Status
0.4.0-alpha.1integrity.jsthin-hook plugin andintegrityService.jsexpress middleware;validationService.jsforClientIntegrity.browserHashvalidationcryptolibraryonly-if-cachedfetchingbody-parsercan process a request body only once per requestbody-parsers process decrypted body byintegrityService.jsmiddleware? Any tricks?http-proxy-middlewareneeds a patch forproxyReqevents (working for/errorReport.jsonPOST requests)express.staticandexpressStaticGzipwork well (non-POST, no request body)postHtml.jsneeds a trick to parse the decryptedreq.bodyinapplication/x-www-form-urlencodedformat for itself like thisdecodeURIComponent(new URL('https://localhost/?' + req.body.toString()).searchParams.get('type'))node-addon-apifor more than 40 times acceleration from node-forge/integrity307 Redirect toon errorsabout:blankResponse.error()to avoid vulnerabilities in iframewhitelist.jsonto specify the list of URL paths that can be retrieved without checking integrity at the serverblacklist.jsonto specify the list of URL paths that cannot be retrievedindex.htmlfor the entry pageupgradeevent is fired on error responses so that the app can suspend for upgradingupgradeevents must be dispatched to all the tabs of the appsuspendprocess required when the server is unresponsive and the session has expired?app.all('/*', express.static(dir))with 4 workers)app.all('/*', express.static(dir))with 4 workers as a back end for http2 nginx reverse proxy)Encrypted Response
Timeline
Request Headers
Response Headers