HSTS preload list

[baknu]

Mail discussion between RD and BK on 2018-09-06:

This is about the Chrome preload list (https://hstspreload.org/) that is also used by other major browsers but probably not by all browsers.

In the HSTS RFC there is also a paragraph on "HSTS Pre-Loaded List" (https://tools.ietf.org/html/rfc6797#section-12.3).

Question is whether Internet.nl should check this list when performing the HSTS subtest?

[WKobes]

Useful API: https://hstspreload.org/api/v2/preloadable?domain=example.nl https://hstspreload.org/api/v2/status?domain=example.nl

[bwbroersma]

Mozilla :fox_face: Firefox uses it's on HSTS preload list, seeded on the Chrome HSTS preload list. See Mozilla Wiki: SecurityEngineering/HTTP Strict Transport Security (HSTS) Preload List:

The xpcshell script is here. Output from the automated job as run on each branch is available here: mozilla-central esr60 (scroll down until there's a line containing "pfu", click on that, then click on "live.log" in the pane that pops up).

Current list:

• :fox_face: Firefox: nsSTSPreloadList.inc [raw] (2.8M) Note: the second column 0 or 1 indicates includeSubdomains. Currently 7543 (4.8%) domains of 156460 are stored without includeSubdomains:

  awk '$2=="0"{nic++}END{print nic}' nsSTSPreloadList.inc

  It uses Deterministic Acyclic Finite State Automata see Bug 1382001 - Use an even more efficient format for kSTSPreloadList that converts the inc file via make_dafsa.py. Todo: check if Mozilla really 'accepts' all TLD/PSD preloads of Chrome.

• Chrome: transport_security_state_static.json (14MB json) Can be downloaded via the tgz or as base64. Using the tgz is 10 times more efficient in bandwidth.

  # currently downloads 1.8MB:
  curl -sSfA '' --compressed 'https://chromium.googlesource.com/chromium/src/net/+archive/refs/heads/main/http.tar.gz' | bsdtar -Oxf- "transport_security_state_static.json"
  # currently downloads 18MB:
  curl -sSfA '' --compressed 'https://chromium.googlesource.com/chromium/src/net/+/refs/heads/main/http/transport_security_state_static.json?format=TEXT' | base64 -d

  Currently 134 (0.1%) domains of 123084 are stored without include_subdomains:

   sed '/^\s*\/\//d'| jq '.entries[]|select(.include_subdomains==false)|.name' -r | wc -l

• Edge?
• Safari?

Both Firefox and Chrome have, next to the tld preloads, also quite some nic.tld preloads (e.g. both esq and nic.esq).

I don't know if the lists are only baked in, or also requested by clients out of band with updates. In any case it should be useful to determine the start/end of a HSTS preload, and the specific versions it's in.

BTW it's interesting to see for these files the Chrome changelog looks way more manual (see also their manual options) than the :fox_face: Mozilla Firefox changelog.