cisagov / scanner

Automated pshtt, trustymail, and sslyze scanning
Creative Commons Zero v1.0 Universal
24 stars 5 forks source link

pshtt scans failed this weekend #67

Closed jsf9k closed 1 year ago

jsf9k commented 1 year ago

🐛 Summary

The pshtt scans did not take place this weekend.

To reproduce

  1. Attempt to run a full BOD 18-01 scan.
  2. Note that no pshtt AWS Lambdas are actually launched.

Expected behavior

The pshtt scans should complete without issue, as they have since time immemorial.

Any helpful log output or screenshots

Here is the error I am seeing in the Docker logs of the scanner container:

[pshtt] Downloading third party data...

-------------------------

Fetching Chrome preload list from source...

Starting new HTTPS connection (1): chromium.googlesource.com:443
https://chromium.googlesource.com:443 "GET /chromium/src/net/+/master/http/transport_security_state_static.json?format=TEXT HTTP/1.1" 404 None
Traceback (most recent call last):
  File "/home/scanner/domain-scan/scan", line 622, in <module>
    run(options, unknown, cache_dir, results_dir)
  File "/home/scanner/domain-scan/scan", line 122, in run
    scan_domains(scans, domains, options)
  File "/home/scanner/domain-scan/scan", line 171, in scan_domains
    init = scanner.init(environment, options)  # type: ignore
  File "/home/scanner/domain-scan/scanners/pshtt.py", line 41, in init
    'preload_list': pshtt.load_preload_list(),
  File "/usr/local/lib/python3.6/site-packages/pshtt/pshtt.py", line 1485, in load_preload_list
    raw = base64.b64decode(raw).decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe4 in position 1: invalid continuation byte
jsf9k commented 1 year ago

Looks like the issue is simply that this URL has changed. This is the new URL.