youtube not blocked but doesn't load

[ohnonot]

In all my browsers, I'm getting "Server not found" or "server DNS address could not be found" or some such for youtube.com.

youtube.com is not blocked in /etc/hosts, and neither are google.com or ytimg.com. i also tried unblocking s.youtube.com and s2.youtube.com.

i can ping youtube.com just fine.

after I do

mv /etc/hosts /etc/hosts~ && cp /etc/hostsblock/hosts.head /etc/hosts

youtube.com loads (even with all my browser addons, e.g. noscript, active).

how can I troubleshoot this? nothing in /etc/hosts or /etc/hostsblock/* seems to hint at youtube being blocked. i tried the network tab of palemoon's and brave's developer tools, but the only network request goes out to youtube.com itself. what is youtube looking for?

[gaenserich]

Also see what hostsblock-urlcheck "http://www.youtube.com" tells you. A block list might have gotten too aggressive and blocked a support domain, e.g. yimg.com.

Am 21.12.2016 03:13 schrieb "ohnonot" notifications@github.com:

In all my browsers, I'm getting "Server not found" or "server DNS address could not be found" or some such for youtube.com.

youtube.com is not blocked in /etc/hosts, and neither are google.com or ytimg.com. i also tried unblocking s.youtube.com and s2.youtube.com.

i can ping youtube.com just fine.

after I do

mv /etc/hosts /etc/hosts~ && cp /etc/hostsblock/hosts.head /etc/hosts

youtube.com loads (even with all my browser addons, e.g. noscript, active).

how can I troubleshoot this? nothing in /etc/hosts or /etc/hostsblock/* seems to hint at youtube being blocked. i tried the network tab of palemoon's and brave's developer tools, but the only network request goes out to youtube.com itself. what is youtube looking for?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/gaenserich/hostsblock/issues/65, or mute the thread https://github.com/notifications/unsubscribe-auth/ACHno0C1jy8wN5ylC0kmenAPVyizT9W_ks5rKN_EgaJpZM4LSrpZ .

[ohnonot]

i was actually grepping /etc/hosts itself for youtube.com.

is yimg.com another youtube support domain?

i was grepping for ytimg.com, and that gave only 2 results (and unblocking those did not help), but yimg.com gives quite a few results.

according to this answer .ytimg.com is part of the youtube network, but not .yimg.com.

i could test that myself, however i am now a little unclear how to use the hostsblock-urlcheck utility!

example: hostsblock-urlcheck googlevideo.com => NOT BLOCKED

grep googlevideo.com /etc/hosts still gives results.

hostsblock-urlcheck *googlevideo.com => BLOCKED - my choice: 2) Unblock/unredirect all sites containing url *googlevideo.com

grep googlevideo.com /etc/hosts still gives results.

running sudo hostsblock, grepping /etc/hosts again for googlevideo.com, i still get the exact same results.

what am i doing wrong?

[gaenserich]

(P.S. make sure to restart dnsmasq or systemd-resolved to flush out the old blocklists from memory, if you haven't already done that automatically through the hostsblock-urlcheck script or hostsblock itself)

The next step would be to monitor your connection with youtube.com with tcpdump. Install tcpdump, turn off your blocking, start tcpdump with this command:

$ sudo tcpdump

And then visit youtube and see what tcpdump spits out. Then, turn blocking back on and run tcpdump as before to see what (doesn't) come up. That may give you a better feel for what's missing.

Jacob A. van der Kolk http://personal.psu.edu/jav209 Instructor of German, ABD jakevanderkolk@protonmail.com (PGP Public Key http://personal.psu.edu/jav209/jakevanderkolk_protonmail.asc)

2016-12-22 1:52 GMT-05:00 ohnonot notifications@github.com:

i was actually grepping /etc/hosts itself for youtube.com.

is yimg.com another youtube support domain?

i was grepping for ytimg.com, and that gave only 2 results (and unblocking those did not help), but yimg.com gives quite a few results.

according to this answer https://stackoverflow.com/a/29133982 . ytimg.com is part of the youtube network, but not .yimg.com.

i could test that myself, however i am now a little unclear how to use the hostsblock-urlcheck utility!

example: hostsblock-urlcheck googlevideo.com => NOT BLOCKED

grep googlevideo.com /etc/hosts still gives results.

hostsblock-urlcheck googlevideo.com => BLOCKED - my choice: 2) Unblock/unredirect all sites containing url googlevideo.com

grep googlevideo.com /etc/hosts still gives results.

running sudo hostsblock, grepping /etc/hosts again for googlevideo.com, i still get the exact same results.

what am i doing wrong?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/gaenserich/hostsblock/issues/65#issuecomment-268731550, or mute the thread https://github.com/notifications/unsubscribe-auth/ACHno1nabSjUgrgnGjBmu9VEpIR6Wszxks5rKh4pgaJpZM4LSrpZ .

[ohnonot]

found the culprit with tcpdump: youtube-ui.l.google.com - after unblocking that (and none of the previously mentioned) youtube loads again.

i found out that i have to manually re-run hostsblock each time I use hostsblock-urlcheck, otherwise the just unblocked domain still shows up as blocked:

# hostsblock-urlcheck youtube-ui.l.google.com
Checking to see if url is blocked or unblocked...

'youtube-ui.l.google.com' BLOCKED by blocklist(s) /etc/hosts.block.old
    1) Unblock/unredirect just youtube-ui.l.google.com
    2) Unblock/unredirect all sites containing url youtube-ui.l.google.com
    3) Keep blocked/redirected
1-3 (default: 3): ^C

this is not by design, i take? why is it happening?

my /etc/hostsblock.conf currently looks like this:

# awk '!/^ *#/ && NF' hostsblock.conf
cachedir="/var/cache/hostsblock" # DEFAULT
tmpdir="/dev/shm" # DEFAULT
hostsfile="/etc/hosts" # DEFAULT. If not using a dns caching daemon
redirecturl="0.0.0.0"
redirects="0" # DEFAULT
hostshead="/etc/hostsblock/hosts.head" # DEFAULT
blacklist="/etc/hostsblock/black.list"
whitelist="/etc/hostsblock/white.list"
connect_timeout=60 # DEFAULT
retry=1
backup_old=1
recycle_old=1 # DEFAULT
verbosity=3
annotate=/var/lib/hostsblock.db.gz
blocklists=(
'http://winhelp2002.mvps.org/hosts.zip'         # anti-ad and anti-malware list (winhelp2002.mvps.org/hosts.htm)
'http://pgl.yoyo.org/as/serverlist.php?hostformat=hosts&mimetype=plaintext' # anti-ad (pgl.yoyo.org/adservers/)
'http://hosts-file.net/download/hosts.zip'      # hpHosts main anti-ad and anti-malware (hosts-file.net/)
'http://www.malwaredomainlist.com/hostslist/hosts.txt'  # anti-malware (www.malwaredomainlist.com/)
'http://hosts-file.net/ad_servers.txt'          # anti-ads-only version of hpHosts main list (hosts-file.net)
'http://someonewhocares.org/hosts/hosts'        # anti-ad and anti-malware (someonewhocares.org/hosts/)
'http://adaway.org/hosts.txt' # blocking mobile ad and anti-analytics (adaway.org)
'http://hosts-file.net/hphosts-partial.asp'     # hpHosts inter-release (hosts-file.net/)
'http://hostsfile.org/Downloads/BadHosts.unx.zip'   # anti-malware (hostsfile.org/hosts.html)
'http://hostsfile.mine.nu/Hosts.zip'            # anti-ad (hostsfile.mine.nu)
'http://sysctl.org/cameleon/hosts'          # anti-ad (sysctl.org/cameleon/)
)

and manually running hostsblock spits out this:

 hostsblock
[INFO] Checking blocklists for updates...
[INFO] Refreshed blocklist http://winhelp2002.mvps.org/hosts.zip.
[INFO] Refreshed blocklist http://pgl.yoyo.org/as/serverlist.php?hostformat=hosts&mimetype=plaintext.
[INFO] Refreshed blocklist http://hosts-file.net/download/hosts.zip.
[INFO] Refreshed blocklist http://www.malwaredomainlist.com/hostslist/hosts.txt.
[INFO] Refreshed blocklist http://hosts-file.net/ad_servers.txt.
[INFO] Refreshed blocklist http://someonewhocares.org/hosts/hosts.
[NOTE] CHANGES FOUND for blocklist http://someonewhocares.org/hosts/hosts.
[INFO] Refreshed blocklist http://adaway.org/hosts.txt.
[INFO] Refreshed blocklist http://hosts-file.net/hphosts-partial.asp.
[INFO] Refreshed blocklist http://hostsfile.org/Downloads/BadHosts.unx.zip.
[INFO] Refreshed blocklist http://hostsfile.mine.nu/Hosts.zip.
[INFO] Refreshed blocklist http://sysctl.org/cameleon/hosts.
[INFO] Changes found among blocklists. Extracting and preparing cached files to working directory...
[INFO] Unzipped hostsfile.mine.nu.Hosts.zip.
[INFO] Unzipped hostsfile.org.Downloads.BadHosts.unx.zip.
[INFO] Unzipped hosts-file.net.download.hosts.zip.
[INFO] Unzipped winhelp2002.mvps.org.hosts.zip.
[INFO] Recycling old /etc/hosts into new version...
[INFO] Recycled old /etc/hosts into new version.
[INFO] Backed up old /etc/hosts.
[INFO] Replaced existing /etc/hosts with .
[INFO] Compiling block entries into /etc/hosts...
[INFO] Compiled block entries into /etc/hosts.
[INFO] Appending blacklisted entries to /etc/hosts...
[INFO] Appended blacklisted entries to /etc/hosts.
[INFO] /etc/hosts: 654910 urls redirected to 0.0.0.0.
[INFO] /etc/hosts: 1 urls redirected to 127.0.0.1   xxxxxxxx.
[INFO] /etc/hosts: 1 urls redirected to 127.0.0.1   xxxxxxxxxx xxxxxxxxxx.
[INFO] Executing postprocessing...
[INFO] Postprocessing completed.
[NOTE] Cleaned up /dev/shm/hostsblock.
[INFO] DONE.

[ohnonot]

i think this should be closed?