pirate / sites-using-cloudflare

:broken_heart: Archived list of domains using Cloudflare DNS at the time of the CloudBleed announcement.
1.92k stars 318 forks source link

Automated verification #215

Closed Zenexer closed 7 years ago

Zenexer commented 7 years ago

Status

Purpose

We're attempting to verify domains that were using the Cloudflare proxy during the relevant timeframe. To do this, we'll start by running our existing lists against proxy checks; any domain is currently using the proxy was almost certainly using it during the incident timeframe, so we can consider those verified.

We can't rule out any domains this way, as many sites have switched away from Cloudflare since the incident. However, we'll have significantly fewer domains that we need to check manually or run more aggressive checks against.

An important goal of this mini-project will be to generate reproducible results, meaning that anyone with the same input data will be able to produce the same output. This won't be perfect, as DNS entries will change, but other developers will at least be able to scrutinize and test our process.

This mini-project will not eliminate domains with static content; such domains will be kept on the verified domain list. It's not our job to assess the severity of leaks for any specific site; that's left to the end user. The "static content" rule was too inaccurate in assessing severity; for example, it's possible that records of visits to static content could endanger lives in the case of political activism under oppressive governments. We're recording affected sites, not whether there's a relevant impact on the end user.

Can I create a website that allows users to search this list?

That would be unadvisable. Sites created to search the existing list were often poorly made and therefore inaccurate. While this new list will make such errors more difficult, we're still not producing data that's meant for direct consumption by an end user--that's why it's on GitHub, a site for developers. Site owners can search the list and choose how to notify their respective users, or end users that happen to be knowledgeable enough to search the database themselves can choose to raise awareness for specific sites of their choosing.

It's possible that we may eventually gather enough information to recommend direct access to this database by end users. However, we are not currently at that point.

pirate commented 7 years ago

+++ on all points. This list was never intended for end users, and I don't like that it's gotten as much non-tech press as it has. I'm vaguely considering closing the issue on search tools and making the wiki harder to find for the same reason. The original purpose was to encourage people to reset passwords, but really the important effect was to put pressure on affected vendors to rotate session keys, and publish blog posts on whether or not they were impacted.

I agree that we should probably stop manually checking sites for static content, as this is an inherently flawed process prone to human error. Unfortunately it's hard to stop doing this without being unfair to new removal requests. I've gone into some more detail on why I'm requiring ownership verification for large sites here: https://github.com/pirate/sites-using-cloudflare/issues/213, it's not a perfect process, the aim is just to cover my bases as far as liability goes.

pirate commented 7 years ago

Archived the list, no longer doing manual verification.