Open xhdix opened 3 years ago
wkrp
commented
3 years ago You say that the 503 injection is temporary. How long does it take to stop happening? Does it happen on the first request, and not happen on the second? Is it time-based?
To me, it almost looks like a transparent HTTP proxy with a genuine malfunction.
xhdix
commented
3 years ago Until recently, it was temporary. And most of the time it only happened at the first request or only for up to 5 minutes. But in the case of ampproject.org it is permanent.
And now I see that the behavior of the censorship system has become much worse: https://twitter.com/alirezashirazi/status/1291308509951336448
wkrp
commented
3 years ago And now I see that the behavior of the censorship system has become much worse: https://twitter.com/alirezashirazi/status/1291308509951336448
اختلال نت برخی سرویس دهندگتن اینترنت در کشور... گاهی صفحه لود میشه گاهی نمیشه و گاهی صفحه فیلتر نمایش داده میشه! (تست روی اینترنت مخابرات استان تهران)
Net disruption of some internet service providers in the country ... sometimes the page is loaded, sometimes it is not and sometimes the filter page is displayed! (Test on Tehran Telecommunication Internet)
That's interesting. It's also slightly different behavior than what you experienced. The error page is just Service unavailable, not <H1>503 Service Unavailable</H1>. Also, the video shows that sometimes the page returned is not a 503, but the usual Iran 403, i.e., the one that has <iframe src="http://10.10.34.34?type=...&policy=MainPolicy " style="width: 100%; height: 100%" scrolling="no" marginwidth="0" marginheight="0" frameborder="0" vspace="0" hspace="0"></iframe>.
https://twitter.com/alirezashirazi/status/1291363670673547264
دقیقا معلوم نیست اما احتمالا از سیستم کش سرویس دهنده است
It is not known exactly, but it is probably from the server cache system
This explanation seems plausible to me.
xhdix
commented
3 years ago https://twitter.com/alirezashirazi/status/1291363670673547264
دقیقا معلوم نیست اما احتمالا از سیستم کش سرویس دهنده است
It is not known exactly, but it is probably from the server cache system
This explanation seems plausible to me.
(Blogfa belongs to @alirezashirazi.)
Better translation: It is not clear perfectly, but it is probably from the service provider cache system
xhdix
commented
3 years ago New case:
In TCI :
In MCI:
I hope the tests will be available here soon: https://explorer.ooni.org/search?since=2020-08-10&until=2020-08-12&probe_cc=IR&test_name=web_connectivity&domain=get.videolan.org
xhdix
commented
3 years ago Today a friend sent me a pcap from his web browsing which shows that all requests to detectportal.firefox.com , ocsp.pki.goog and ocsp.int-x3.letsencrypt.org are similarly affected by the censorship system.
xhdix
commented
3 years ago Today's test with OONI Probe-cli on TCI (AS58224) :
1. http://4genderjustice.org/
2. http://666games.net/
3. http://8thstreetlatinas.com/
4. http://abc.go.com/
5. http://abpr2.railfan.net/
6. http://adultfriendfinder.com/
7. http://advocacy.globalvoicesonline.org/
8. http://alt.com/
9. http://amphetamines.com/
10. http://amygoodloe.com/
11. http://anon.inf.tu-dresden.de/
12. http://anonymizer.secuser.com/
13. http://bit.ly/
14. http://bittornado.com/
15. http://blogspot.com/
16. http://blueskyswimwear.com/
17. http://bnaibrith.org/
18. http://bravotube.net/
19. http://cocaine.org/
20. http://comohacerseunaborto.com/
21. http://cultdeadcow.com/
22. http://dextroverse.org/
23. http://download.cnet.com/
24. http://ecstasy.org/
25. http://emailaddresses.com/
26. http://exscn.net/
27. http://gaytoday.com/
28. http://genderandaids.unwomen.org/
29. http://go.com/
30. http://godhatesfags.com/
31. http://guardster.com/
32. http://hackers.com/
33. http://hightimes.com/
34. http://ilga.org/
35. http://imo.im/
36. http://instinctmagazine.com/
37. http://international.ohmynews.com/
38. http://islamonline.net/
39. http://jainworld.com/
40. http://justicewomen.com/
41. http://kickassclassical.com/
42. http://kidshealth.org/
43. http://lauraannjacobs.com/
44. http://lgbt.foundation/
45. http://norml.org/
46. http://occupystreams.org/
47. http://proxy.org/
48. http://proxytools.sourceforge.net/
49. http://russia.tv/
50. http://seclists.org/
51. http://secondlife.com/
52. http://sfsi.org/
53. http://sierraclub.org/
54. http://tdov.org/
55. http://thepiratebay.org/
56. http://timesofindia.indiatimes.com/
57. http://translation.langenberg.com/
58. http://transsexual.org/
59. http://tvants.uptodown.com/
60. http://twilight.ws/
61. http://twitter.com/
62. http://ultrasurf.us/
63. http://weblogs.us/
64. http://wedo.org/
65. http://weedfarmer.com/
66. http://whitehonor.com/
67. http://womeninblack.org/
68. http://www.150m.com/
69. http://www.2ddepot.com/
70. http://www.4chan.org/
71. http://www.4online-gambling.com/
72. http://www.4shared.com/
73. http://www.888casino.com/
74. http://www.911memorial.org/
75. http://www.911truth.org/
76. http://www.abortionno.org/
77. http://www.absinth.com/
78. http://www.aceshigh.com/
79. http://www.acquisitionx.com/
80. http://www.advocate.com/
81. http://www.advocatesforyouth.org/
82. http://www.af.mil/
83. http://www.afterellen.com/
84. http://www.aidsalliance.org/
85. http://www.aleph.to/
86. http://www.americannaziparty.com/
87. http://www.angryharry.com/
88. http://www.animalliberationfront.com/
89. http://www.anonymitychecker.com/
90. http://www.appzplanet.com/
91. http://www.arabrenewal.com/
92. http://www.arabtimes.com/
93. http://www.asterisk.org/
94. http://www.atimes.com/
95. http://www.auduboninternational.org/
96. http://www.babylon-x.com/
97. http://www.barmeister.com/
98. http://www.beerinfo.com/
99. http://www.benedelman.org/
100. http://www.betfair.com/
101. http://www.birthcontrol.com/
102. http://www.biz.ly/
103. http://www.blackhat.be/
104. http://www.blackjackinfo.com/
105. http://www.blogeasy.com/
106. http://www.blogsome.com/
107. http://www.btselem.org/
108. http://www.buddhanet.net/
109. http://www.cannabis.info/
110. http://www.carnivalcasino.com/
111. http://www.casinotropez.com/
112. http://www.centcom.mil/
113. http://www.chantelle.com/
114. http://www.childrensdefense.org/
115. http://www.cidh.org/
116. http://www.connotea.org/
117. http://www.copticchurch.net/
118. http://www.coquette.com/
119. http://www.crazyshit.com/
120. http://www.cseindia.org/
121. http://www.dailymotion.com/
122. http://www.darknet.org.uk/
123. http://www.darpa.mil/
124. http://www.datpiff.com/
125. http://www.democracycaucus.net/
126. http://www.dharmanet.org/
127. http://www.dia.mil/
128. http://www.dit-inc.us/
129. http://www.download.com/
130. http://www.drudgereport.com/
131. http://www.drugsense.org/
132. http://www.earthaction.org/
133. http://www.efonica.com/
134. http://www.eluniversal.com/
135. http://www.episcopalrelief.org/
136. http://www.eurogrand.com/
137. http://www.euthanasia.cc/
138. http://www.exgay.com/
139. http://www.exmormon.org/
140. http://www.familiesaretalking.org/
141. http://www.familycareintl.org/
142. http://www.feedtheminds.org/
143. http://www.feminist.org/
144. http://www.fepproject.org/
145. http://www.fgmnetwork.org/
146. http://www.fondationdefrance.org/
147. http://www.foreignword.com/
148. http://www.formercatholic.com/
149. http://www.frc.org/
150. http://www.freeexpression.org/
151. http://www.freehomepage.com/
152. http://www.freespeech.com/
153. http://www.fring.com/
154. http://www.fuckingfreemovies.com/
155. http://www.gamingday.com/
156. http://www.gay.com/
157. http://www.gayhealth.com/
158. http://www.gearthblog.com/
159. http://www.getdrupe.com/
160. http://www.ghostrecon.com/
161. http://www.giganews.com/
162. http://www.ginvodka.org/
163. http://www.glil.org/
164. http://www.globalfire.tv/
165. http://www.globalr2p.org/
166. http://www.goarch.org/
167. http://www.grandonline.com/
168. http://www.hackforums.net/
169. http://www.hackhull.com/
170. http://www.hanes.com/
171. http://www.hivandhepatitis.com/
172. http://www.hon.ch/
173. http://www.hrcr.org/
174. http://www.hrea.org/
175. http://www.http-tunnel.com/
176. http://www.ifeminists.com/
177. http://www.ifge.org/
178. http://www.ifj.org/
179. http://www.ihf-hr.org/
180. http://www.ihr.org/
181. http://www.iicwc.org/
182. http://www.ilhr.org/
183. http://www.infowar-monitor.net/
184. http://www.interactworldwide.org/
185. http://www.isiswomen.org/
186. http://www.iskcon.com/
187. http://www.islameyat.com/
188. http://www.islamicity.org/
189. http://www.itsyoursexlife.com/
190. http://www.iwantim.com/
191. http://www.jdl.org/
192. http://www.jesussaves.cc/
193. http://www.jewwatch.com/
194. http://www.jmarshall.com/
195. http://www.jsf.mil/
196. http://www.judaismconversion.org/
197. http://www.kazaa.com/
198. http://www.kcna.kp/
199. http://www.keptprivate.com/
200. http://www.khrp.org/
201. http://www.kurtuluscephesi.com/
202. http://www.laborrightsnow.org/
203. http://www.lasenza.com/
204. http://www.lesbiansubmission.com/
205. http://www.lingerieatlarge.com/
206. http://www.lingo.com/
207. http://www.luckynugget.com/
208. http://www.luwaran.net/
209. http://www.lyricwiki.org/
210. http://www.mail.lycos.com/
211. http://www.mail2web.com/
212. http://www.marijuana.com/
213. http://www.match.com/
214. http://www.mizzima.com/
215. http://www.muhammadanism.com/
216. http://www.mywebcalls.com/
217. http://www.navy.mil/
218. http://www.nazi-lauck-nsdapao.com/
219. http://www.nclrights.org/
220. http://www.neonjoint.com/
221. http://www.netaddress.com/
222. http://www.netzoola.com/
223. http://www.no-porn.com/
224. http://www.oic-oci.org/
225. http://www.oicc.org/
226. http://www.omct.org/
227. http://www.oneworld.net/
228. http://www.onlinedating.com/
229. http://www.onlinewomeninpolitics.org/
230. http://www.oovoo.com/
231. http://www.orthodoxconvert.info/
232. http://www.pacom.mil/
233. http://www.partypoker.com/
234. http://www.pc2call.com/
235. http://www.pcusa.org/
236. http://www.pdhre.org/
237. http://www.peacefire.org/
238. http://www.phenoelit.org/
239. http://www.playboy.com/
240. http://www.pof.com/
241. http://www.poker.com/
242. http://www.pokerpages.com/
243. http://www.pornhub.com/
244. http://www.positive.org/
245. http://www.postcards-for-iran.org/
246. http://www.pravda.ru/
247. http://www.prolife.com/
248. http://www.prophetofdoom.net/
249. http://www.proxyweb.net/
250. http://www.quantico.marines.mil/
251. http://www.queernet.org/
252. http://www.ran.org/
253. http://www.realbeer.com/
254. http://www.religiousconsultation.org/
255. http://www.religioustolerance.org/
256. http://www.repubblica.com/
257. http://www.riftgame.com/
258. http://www.righttodie.ca/
259. http://www.riverbelle.com/
260. http://www.roxypalace.com/
261. http://www.royalvegas.com/
262. http://www.ruf-ch.org/
263. http://www.satp.org/
264. http://www.sbc.net/
265. http://www.scarleteen.com/
266. http://www.schwarzreport.org/
267. http://www.sealswcc.com/
268. http://www.securenym.net/
269. http://www.securityfocus.com/
270. http://www.securitytracker.com/
271. http://www.sexandu.ca/
272. http://www.sexedlibrary.org/
273. http://www.shinto.org/
274. http://www.sida.se/
275. http://www.slsknet.org/
276. http://www.socom.mil/
277. http://www.solicitorsfromhell.com/
278. http://www.sos-reporters.net/
279. http://www.southcom.mil/
280. http://www.speeddater.co.uk/
281. http://www.spinpalace.com/
282. http://www.sportingbet.com/
283. http://www.stopstreetharassment.org/
284. http://www.stratcom.mil/
285. http://www.talkyou.me/
286. http://www.tango.me/
287. http://www.teenhealthfx.com/
288. http://www.terrorismfiles.org/
289. http://www.thegooddrugsguide.com/
290. http://www.thehacktivist.com/
291. http://www.tialsoft.com/
292. http://www.tobacco.org/
293. http://www.topcities.com/
294. http://www.topdrawers.com/
295. http://www.towleroad.com/
296. http://www.truthnet.org/
297. http://www.ucc.org/
298. http://www.ultimate-anonymity.com/
299. http://www.ultimatebirthcontrol.com/
300. http://www.unfpa.org/
301. http://www.upci.org/
302. http://www.usacasino.com/
303. http://www.usafa.af.mil/
304. http://www.uscg.mil/
305. http://www.vanguardnewsnetwork.com/
306. http://www.venus.com/
307. http://www.voanews.com/
308. http://www.voicecommercegroup.com/
309. http://www.volcanomail.com/
310. http://www.warchild.org/
311. http://www.warhammeronline.com/
312. http://www.wcicc.org/
313. http://www.webbox.com/
314. http://www.well.com/
315. http://www.whitepower.com/
316. http://www.wiesenthal.com/
317. http://www.wluml.org/
318. http://www.womensmediacenter.com/
319. http://www.womensmediapool.org/
320. http://www.worldhealth.net/
321. http://www.worldlingo.com/
322. http://www.worldrtd.net/
323. http://www.wzo.org.il/
324. http://www.xinhuanet.com/
325. http://www.xroxy.com/
326. http://www.xvideos.com/
327. http://www.ymca.int/
328. http://www.youporn.com/
329. http://www3.iaisite.org/
(edit: scheme added)
wkrp
commented
3 years ago Today's test with OONI Probe-cli on TCI (AS58224) :
Is this a typical result, or was there more or less blocking than usual in this measurement?
Looking at citizenlab/test-lists, there are 2276 domains in the global+ir list. The 329 you documented therefore constitute about 15%.
$ wc -l lists/global.csv lists/ir.csv
1446 lists/global.csv
830 lists/ir.csv
2276 totalThe seemingly random selection of domains makes me think that the 503s are a random or transient failure in the filter boxes, not targeted at these domains specifically.
I notice ampproject.org is not in the list. Do you know, is that domain consistently or inconsistently blocked with 503?
xhdix
commented
3 years ago A typical result. They were HTTP URLs that received a specific 503 error. e.g. : https://github.com/citizenlab/test-lists/blob/fd20da4cca47a0767d08ad462adaf8e1d9d3ad48/lists/global.csv#L3
Also, these HTTP URLs did not receive such an error:
1. http://btggaming.com/
2. http://care.org/
3. http://earthwatch.org/
4. http://fteproxy.org/
5. http://insecure.org/
6. http://lambdalegal.org/
7. http://peacefire.org/
8. http://ww1.lirio.us/
9. http://www.cbsnews.com/
10. http://www.clubdicecasino.com/
11. http://www.earthwatch.org/
12. http://www.gamespot.com/
13. http://www.godalone.org/
14. http://www.guerrillagirls.com/
15. http://www.harkatulmujahideen.org/
16. http://www.islamic-relief.com/
17. http://www.last.fm/
18. http://www.learningpartnership.org/
19. http://www.lycos.com/
20. http://www.naral.org/
21. http://www.ned.org/
22. http://www.siecus.org/
23. http://www.sina.com.cn/
24. http://www.teensource.org/
25. http://www.theepochtimes.com/
26. http://www.typepad.com/
27. http://www.wikia.com/
There were also 51 packet injection case in HTTPS URLs. (Which was mentioned a little in #39 in the past.)
HTTPS://cdn.ampproject.org/ is also accessible:
https://github.com/citizenlab/test-lists/blob/fd20da4cca47a0767d08ad462adaf8e1d9d3ad48/lists/global.csv#L1139
$ curl http://ampproject.org
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>302 Moved</TITLE></HEAD><BODY>
<H1>302 Moved</H1>
The document has moved
<A HREF="https://ampproject.org/">here</A>.
</BODY></HTML>$ curl http://cdn.ampproject.org
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>302 Moved</TITLE></HEAD><BODY>
<H1>302 Moved</H1>
The document has moved
<A HREF="https://cdn.ampproject.org/">here</A>.
</BODY></HTML>
If a site that is only HTTP and is new or has not been used by the user for a long time, it will encounter an error exactly as follows:
HTTP Header Field ManipulationTest : https://explorer.ooni.org/measurement/20190428T192258Z_AS197207_py2wAHgNm3shTTH8lgGkkqqbt2k2StQKiva5vH96JQ5zrNy49H https://explorer.ooni.org/measurement/20190402T090256Z_AS58224_X2HTqiXeWO1Xb9NJxKsG6ln4v1OfID2zl0BvcdA3QzVXHfn8LpXref: https://github.com/ooni/probe/issues/911
At first, this situation only happened in some random circumstances. For example, if the user requested some/a lot of unauthorized sites and the problem was solved after about 5 minutes. But now the situation is worse and it happens more often in most ISPs.
Web ConnectivityTest : https://explorer.ooni.org/measurement/20200526T152428Z_AS31549_xURYxcuoJHgcRMs3p9xReVMz5tO4mTZdNENjyok4UpZOJ3buaP?input=http%3A%2F%2Fwww.kernel.org%2F https://explorer.ooni.org/measurement/20200515T155909Z_AS197207_H64C2Juy7lw8yLf5mkJMsCXnW8DsXqKMapsSu0dQelH9evArFv?input=http%3A%2F%2Fwww.kernel.org%2F https://explorer.ooni.org/measurement/20200515T143449Z_AS197207_Gsu6V2zWatXZWr9gRRlugSUxEpFOD7wBsn4NadvvMWKwOfAHDY?input=http%3A%2F%2Fwww.kernel.org%2F https://explorer.ooni.org/measurement/20200420T171706Z_AS58224_mv0hbKXZ6fPlOl4Em51KQlMCS5SB2EigRxWCl8axVMh4mHWeqs?input=http%3A%2F%2Fwww.kernel.org%2F https://explorer.ooni.org/measurement/20200420T140339Z_AS58224_mzgHILUJkX1dyq6iFmXD0p7cR5IH0erEsd9YdguGIXu7H70wMK?input=http%3A%2F%2Fwww.kernel.org%2Fhttps://explorer.ooni.org/measurement/20200805T184723Z_AS197207_KJVNvGgGKsjexP6wy4tLQuERf8XzdWE58WqLGdNgw6OJSKZTfG?input=http%3A%2F%2Ffishgl.com%2F https://explorer.ooni.org/measurement/20200805T184657Z_AS197207_erFUwtej6uR4DnPRcVaLJiKrZxnquRFo3n4bCJbHbI4gM1jeua?input=http%3A%2F%2Ffishgl.com%2F
https://explorer.ooni.org/measurement/20200805T184536Z_AS197207_480IiGKr1oWb2UqKHdXWJTzZIxXKK2oFrDMeWmJ8BZfx5sWwd8?input=http%3A%2F%2Fampproject.org https://explorer.ooni.org/measurement/20200805T184424Z_AS197207_RNLZuiIFK9CMCoafPWByYhKHh9gwiSf9iyeCWTHygJboavMFDj?input=http%3A%2F%2Fampproject.org https://explorer.ooni.org/measurement/20200805T184411Z_AS197207_JTpcCVBnMMmHDmg8KG6gQFqPwzXgXIHjrUt06UoeyjxDW51jDs?input=http%3A%2F%2Fampproject.org
This is important to note because I have seen some censorship circumvention tools consider only HTTP 403 error as blocking. Also, a little bit in Windows and more in Linux, most updates are done via HTTP. In Linux, many apps cannot be installed without a VPN because of this or because of keyword censorship.