openrightsgroup / blocked-org-uk

Template front-end code, markup, style-sheets, images and other assets for the Censorship Monitoring Project (blocked.org.uk)
https://www.blocked.org.uk/
GNU General Public License v3.0
13 stars 5 forks source link

Find .uk suspended domains #221

Closed JimKillock closed 6 years ago

JimKillock commented 6 years ago

We should be able to find .uk domain suspensions at Nominet. See: https://www.infosecurity-magazine.com/news/suspended-uk-domains-double-in-a/

We may also need to find different suspension pages, eg:

DEFRA – Veterinary Medicines Directorate; Metropolitan Police – Fraud and Linked Crime Online (FALCON); the National Crime Agency (NCA); Medicines and Healthcare Products Regulatory Agency (MHRA); Trading Standards; National Fraud Intelligence Bureau (NFIB); and the Police Intellectual Property Crime Unit (PIPCU).

I will put in FoI requests; Dan can you think about why we haven't noticed the suspension pages as yet>

dantheta commented 6 years ago

We've picked up a few - https://www.blocked.org.uk/sites/pipcu

The probe rulesets don't cover the domain seizures at the moment - they don't vary by ISP. Also, during the larger scale URL testing we were only collecting page metadata for sites that had been matched against copyright/parental control. The page data was being collected for the keyword search index, and we didn't have a use for page content of sites that didn't have any blocks registered against them.

JimKillock commented 6 years ago

OK, not a problem. Are we able to detect them now, or to start looking for them?

dantheta commented 6 years ago

Yep - I've got a match pattern for pipcu, and as we find others we can include them too. It might be good to refactor the rules file a bit so that we can have a global pattern list as well as the per-ISP ones.

JimKillock commented 6 years ago

yes, for suspension on one line means it does not need testing elsewhere. The main thing is that we try to record the suspension page.

For PIPCU we could try finding domains that resolve to their IP address 83.138.166.114 (see http://83.138.166.114 ) and then if we like double testing them … except their server isn't responding …

JimKillock commented 6 years ago

See also https://wiki.openrightsgroup.org/wiki/Police_Intellectual_Property_Crime_Unit#Domain_pointing

JimKillock commented 6 years ago

Closing this as we got as far as is likely currently.