search

ooni / pipeline

OONI data processing pipeline
https://ooni.org/
BSD 3-Clause "New" or "Revised" License
40 stars 14 forks source link

Add DNS fingerprints for Iran #390

Closed FedericoCeratto closed 2 years ago

FedericoCeratto commented 2 years ago

Related to https://github.com/ooni/backend/issues/543

hellais commented 2 years ago

While we are at it, I guess we should also add several more DNS fingerprints I have collected:

name,common_name,pattern,location_found,scope,confidence_no_fp,expected_countries,source,exp_url,notes
cl.dns_isp_au_telstra_block_1,,101.167.166.53,dns,isp,10,['AU'],[''],https://explorer.ooni.org/measurement/20220203T193446Z_webconnectivity_AU_1221_n1_E32NLhLwI1WJYE3p?input=http%3A%2F%2Fthepiratebay.org%2F,AU ISP Telstra Block Page
cl.dns_isp_au_telstra_block_2,,101.167.164.53,dns,isp,10,['AU'],[''],https://explorer.ooni.org/measurement/20220203T193446Z_webconnectivity_AU_1221_n1_E32NLhLwI1WJYE3p?input=http%3A%2F%2Fthepiratebay.org%2F,AU ISP Telstra Block Page
cl.dns_isp_au_tpg_block,,202.136.99.184,dns,isp,10,['AU'],['https://www.lifehacker.com.au/2018/06/how-to-bypass-isp-blocking-of-the-pirate-bay-and-other-torrent-sites-for-free/'],https://explorer.ooni.org/measurement/20220202T100548Z_webconnectivity_AU_7545_n1_jIoirPJjohcYOCvM?input=http%3A%2F%2Fthepiratebay.org%2F,AU ISP TPG Block Page
cl.dns_isp_ca_rogers_tva_telus_block,,67.43.226.22,dns,isp,10,['CA'],[''],https://explorer.ooni.org/measurement/20220203T103332Z_webconnectivity_CA_812_n1_7M7ZYtu8kInk4dU3?input=http%3A%2F%2Fapp.atntvv.cc%2F,CA ISP Rogers Telus and TVA Group joint Block Page
cl.dns_isp_ca_shaw_block,,64.59.135.158,dns,isp,10,['CA'],[''],https://explorer.ooni.org/measurement/20220203T170239Z_webconnectivity_CA_6327_n1_3Qts0KFGq9BAcnb1?input=http%3A%2F%2Fdestv.me%2F,Canadian ISP Shaw blockpage
cl.dns_isp_ca_teksavvy_block,,206.248.146.244,dns,isp,10,['CA'],[''],https://explorer.ooni.org/measurement/20220203T004625Z_webconnectivity_CA_5645_n1_cGktKcvMOwO7Tr20?input=http%3A%2F%2Fapp.atntvv.cc%2F,CA ISP Teksavvy Block Page
cl.dns_isp_ru_AS12389,,31.28.24.3,dns,isp,10,['RU'],[''],https://explorer.ooni.org/measurement/20220304T044651Z_webconnectivity_RU_12389_n1_BmRicVwEpc4HG72k?input=https%3A%2F%2Fwww.bbc.com%2F,Serves a blockpage for citytelecom.ru
cl.dns_isp_ru_AS15378,,95.213.158.61,dns,isp,7,['RU'],[''],https://explorer.ooni.org/measurement/20220305T053351Z_webconnectivity_RU_15378_n1_wsIzPXq2OLRTBIAq?input=http%3A%2F%2Fwww.bbc.com,AS of returned IP is mapped to russian hosting provider AS49505 (SELECTEL). Pattern is consistent for several blocked sites.
cl.dns_isp_ru_AS197460,,46.175.31.251,dns,isp,8,['RU'],[''],https://explorer.ooni.org/measurement/20220305T044706Z_webconnectivity_RU_197460_n1_moH7izWEhyf8UJ81?input=https%3A%2F%2Fwww.bbc.com%2Fnews,"Likely used to be a blockpage. The PTR record is host-46-175-31-251.rev.zencom.ru and the AS is AS197460, which is consistent with the network where we observe it. As of 2022-03-05 the session times out when attempting to fetch the index via HTTP (port 80 is open though)."
cl.dns_isp_ru_AS3335,,84.237.49.190,dns,isp,8,['RU'],[''],https://explorer.ooni.org/measurement/20220304T062438Z_webconnectivity_RU_3335_n1_mn3OTB1761hQW6PB?input=https%3A%2F%2Fwww.bbc.com%2F,PTR record host190.49.237.84.nsu.ru and AS is AS3335. As of 2022-03-05 a 503 error is returned when accessing page.
cl.dns_isp_ru_AS35807,,100.64.64.66,dns,isp,7,['RU'],[''],https://explorer.ooni.org/measurement/20220305T121024Z_webconnectivity_RU_35807_n1_QkCl4ZggAoowPpAI?input=http%3A%2F%2Fwww.bbc.com%2Fnews,"Private IP space, but behaviour is consistent on AS35807 for blocked domains."
cl.dns_isp_ru_AS42429,,77.238.226.53,dns,isp,10,['RU'],[''],https://explorer.ooni.org/measurement/20220302T003421Z_webconnectivity_RU_42429_n1_4xzf7tPugdylxlAj?input=https%3A%2F%2Fwww.currenttime.tv%2Ftv%2Fschedule%2F92%2F,Serves a blockpage
cl.dns_isp_ru_AS51547,,80.76.104.20,dns,isp,10,['RU'],[''],https://explorer.ooni.org/measurement/20220304T163214Z_webconnectivity_RU_51547_n1_oUciU7VqaGrmL4HA?input=https%3A%2F%2Fwww.bbc.com%2Fnews%2Fworld-51235105,PTR records are block.tdsplus.ru & balance.tdsplus.ru. We get connection refused when attempting to access it.
cl.dns_isp_ru_AS52207,,188.186.157.49,dns,isp,10,['RU'],[''],https://explorer.ooni.org/measurement/20220305T120459Z_webconnectivity_RU_52207_n1_AlXw32CjmNRv0WxI?input=http%3A%2F%2Fwww.bbc.com%2Fnews,PTR record is k8s-lb-onlyhttp-cluster-ingress.static.cc.ertelecom.ru. Serves blockpage for: http://lawfilter.ertelecom.ru/
cl.dns_isp_ru_AS52207_2,,195.128.72.3,dns,isp,10,['RU'],[''],https://explorer.ooni.org/measurement/20220304T055948Z_webconnectivity_RU_52207_n1_H0I8CD7nFoMYXxCx?input=https%3A%2F%2Fwww.bbc.com%2F,Serves a blockpage
cl.dns_isp_ru_AS60139,,185.77.150.2,dns,isp,10,['RU'],[''],https://explorer.ooni.org/measurement/20220303T120931Z_webconnectivity_RU_60139_n1_RqlkvqFzTf9cySzI?input=https%3A%2F%2Fwww.currenttime.tv%2Ftv%2Fschedule%2F92%2F,Serves a cute cat blockpage
cl.dns_isp_ru_AS8369,,78.29.1.40,dns,isp,10,['RU'],[''],https://explorer.ooni.org/measurement/20220301T122933Z_webconnectivity_RU_8369_n1_M8IHOk8AdoWew7sE?input=https%3A%2F%2Fwww.currenttime.tv%2F,"ASN of the IP is AS8369, which is consistent with the network of the measurement. Connections timeout when attempting to establish a connection on port 80."
cl.dns_isp_ru_AS8427,,188.43.20.67,dns,isp,10,['RU'],[''],https://explorer.ooni.org/measurement/20220305T035626Z_webconnectivity_RU_8427_n1_6zMQbsKYAsOva4L7?input=https%3A%2F%2Fwww.bbc.com%2Frussian%2F,Serves a blockpage for ttk ISP
cl.dns_isp_ru_AS8790,,85.142.29.248,dns,isp,10,['RU'],[''],https://explorer.ooni.org/measurement/20220305T121053Z_webconnectivity_RU_8790_n1_OfI9eozoHc8C4Xkd?input=http%3A%2F%2Fwww.bbc.com%2Fnews,PTR record is block.runnet.ru. We get a blockpage when attempting to access it.
cl.dns_isp_sg_starhub_block,,202.156.3.53,dns,isp,10,['SG'],[''],https://explorer.ooni.org/measurement/20210809T082132Z_webconnectivity_SG_55430_n1_7ImHaJauRVvTo2eF?input=http://www.playboy.com/,Singaporean ISP Starhub blockpage
cl.dns_nat_it_adm_block,,217.175.53.72,dns,nat,10,['IT'],[''],https://explorer.ooni.org/measurement/20211023T083841Z_webconnectivity_IT_30722_n1_hPb937VUDh5Q2NoY?input=http://www.sportingbet.com/,Italian sport/gaming related DNS block
cl.dns_nat_it_agcom_block,,83.224.65.74,dns,nat,10,['IT'],[''],https://explorer.ooni.org/measurement/20211021T193015Z_webconnectivity_IT_30722_n1_Sk3HeZOfHero7Ir9?input=http://gamestorrents.com/,Italian copyright related DNS block
cl.dns_prod_fortidns,,208.91.112.55,dns,prod,10,[],"['https://docs.fortinet.com/document/fortigate/6.0.0/cookbook/124730/troubleshooting', 'https://docs.fortinet.com/product/fortidns/1.3']",https://explorer.ooni.org/measurement/20190404T082635Z_AS6661_WCvp1TnPzuxfEVQl2ZjgPGkDiLXre1Lenrp1cUiwEUJ1oEk6PV?input=https://www.scruff.com/,Fortinet FortiDNS product
cl.dns_prod_opendns_1,,146.112.61.106,dns,prod,10,[],['https://www.opendns.com/cisco-opendns/'],https://explorer.ooni.org/measurement/20170209T174739Z_AS1136_wBIk5bny3acHUuSTuZ8sqtFnKcU47tiRY0LEw08s9m9VDZBsO3?input=http://gayromeo.com,Cisco OpenDNS a popular DNS filtering system. NOTE: there is probably a more sophisticated way to track these IE - CNAME used in the redir.
cl.dns_prod_opendns_2,,208.69.32.164,dns,prod,10,[],['https://www.opendns.com/cisco-opendns/'],https://explorer.ooni.org/measurement/20191007T155258Z_AS20115_DS8uyNobQ0CH9o9fls70ftC1ItOJ7dK5OlMG3YBs7NYrIXQ2Ty?input=https://www.planetromeo.com/,Cisco OpenDNS a popular DNS filtering system. NOTE: there is probably a more sophisticated way to track these IE - CNAME used in the redir.
cl.dns_prod_securely_1,,204.110.220.2,dns,prod,10,[],['https://www.securly.com/'],,DNS filtering product for schools
cl.dns_prod_securely_2,,52.52.63.90,dns,prod,10,[],['https://www.securly.com/'],,DNS filtering product for schools

We should also at some point discuss https://github.com/ooni/backend/issues/516#issuecomment-1036286644