search

18F / pulse

How the federal .gov domain space is doing at best practices and policies.
Other
94 stars 56 forks source link

Domains are missing from the HTTPS and DAP sections #687

Closed gbinal closed 7 years ago

gbinal commented 7 years ago

In the most recent scan (April 22), a number of domains are missing from both the HTTPS and DAP results.

This issue is to track the problem and provide updates until it is resolved.

gbinal commented 7 years ago

Yesterday's scan is now live and the missing domains are back.

We're still investigating what happened. 445 .gov domains failed their April 22 scan. This can happen for a number of reasons, including if a website is not active or is blocking the IP address that the scan is coming from. However it is uncommon for a such a large number of domains to fail like this.

Right now, our best guess is that server running the scan experienced internet connectivity issues and that during that time, the scan was not able to connect with domains in order to complete its scan. A temporary network issue like this would explain the behavior as well as the seemingly random swath of the .gov domain list that failed in the scan. We're continuing to investigate and will update this issue if we discover more.

gbinal commented 7 years ago

Unfortunately, we've not been able to ascertain what exactly happened here. From looking at the logs, our best guess is that something interfered with the scanner's internet access for a period of time, thus causing a set of the scans to fail. Given that it hadn't happened before or since, the hope is that it was a fluke, though if it ever recurs, we may need to escalate our engagement with the cloud services we are using to try and ferret out the issue.