Blocklists being used?

[jenabaivab]

Greetings, Is this project still being developed? I see the commits on Github are dated now. Also, what are the blocklists being used for filtering? OISD I assume? Trying to setup DNSCrypt on Windows and DoT on android from Odisha.

[arapurayil]

Github is used only as the website backend and for tracking issues, which is why you won't find any recent commits. The server is hosted on an aws instance with 'dnsdist' as the load balancer and 'unbound' as the recursive resolver. The blacklist used is 'oisd' with slight modifications. I suggest you select DoH in your DNSCrypt Windows client as I would probably drop support for DNSCrypt protocol in the coming days.

[jenabaivab]

That's great! Set up DoH on YogaDNS windows. Also, what kind of modification are we talking about? I mean let's say I find a false positive. Should I be informing OISD dev on his subreddit or open new issues here?

[jenabaivab]

I saw the ABL repo just now. Couldn't find OISD anywhere :/ The entire point of OISD is to be one single list so that everything is blocked and there are very very few false positives.

[arapurayil]

I'm using OISD and not ABL for dns.arapurayil.com. The modifications I'm talking about are false positives which I had found when using the list, which have since been reported to the dev. You can report false positives directly to the deve here: https://oisd.nl/?p=fp. ABL was used in the server when OISD didn't have an optimized ABP style list. Now that OISD provides an ABP style list I'm using that instead. BTW ABL uses some of the same sources as OISD and also uses the whitelist used in OISD to remove false positives. It haven't included regional sources yet; so some bad domains may slip through which is why I'm using OISD instead. Once I've cleaned up the script and added regional sources, I'd probably switch to ABL.

[jenabaivab]

Ah that's good. Would be really useful if you could mention the blocklist being used in the website FAQ section. That way we can be updated when you switch from OISD to ABL. One more thing, any idea if CNAME cloaking can be avoided via DoT/DoH?

[arapurayil]

That's a good idea. I'm thinking of overhauling the website pretty soon. Will include details about the blocklist and a section for reporting false positives. CNAME is being matched against the blocklist, so even if the tracker employs CNAME cloaking it would still be blocked.

[jenabaivab]

Waiting for the new design. Kinda like the simple design right now though. Something along similar lines without overhauled information would be great. Also, if ABL would be used, a dalse positive section is a compulsion.

[jenabaivab]

@arapurayil Bit weird, but streams on liveonscore.tv don't play when I use your DNS resolver. The moment I switch to Cloudflare or other DNS resolvers, they work as expected, even the resolvers with multiple blocklists in addition to OISD. Please have a look if possible.

[arapurayil]

@jenabaivab Hmm...I couldn't reproduce the issue..might be a ttl issue. Could you try again? I've started another aws instance to be used as the backend server. It will use OISD as the only blocklist. I plan to switch to the new server ~ 02:00 IST 22-01-2020 so as to not cause disruptions. The redesigned website would definitely have a form to report false-positives. I'm also trying to implement an option for the user to check if a particular domain is blocked or not.

[jenabaivab]

@arapurayil same issue again. Tried this stream for instance, same issue. Tested different browsers as well. Works with others like ahadns, clouflare and nextdns(tested just now). http://liveonscore.tv/soccer-streams/premier-league/liverpool-vs-burnley/

EDIT - Please ignore the above. The problem still exists but now I am getting the error with google dns as well. It's a 50/50 chance. I have shifted to another channel for sports. If you still want to get to the root of this, do let me know and I can test anything you need from me.

[arapurayil]

Didn't face the issue...AFAIK ahadns also uses oisd...the only difference being it uses domain style blocking and mine uses adblock style blocking. So it could be that a subdomain is being falsely blocked or the CNAME points to a blocked host. You could try sending me the list of domains being blocked while you're on the page. You can use the browser console for that. Disable browser plugins before you do that though. Lets try if we can get to the bottom of the issue.

[jenabaivab]

@arapurayil Yeah it's working now. Like i said, it's a 50/50 chance. It seems that rule applies for other dns resolvers too, since cloudflare couldn't open the stream early morning today. So i doubt it has anything to do with blocklists.

Btw, do you think some security features of nextdns can be added? You can see so many toggles in their dashboard like typosquatting, dns rebinding and stuff like cache boost too. Anything that can be added will be a bonus, not a necessity I feel. The Web Werks server is phenomenal btw. Definitely snappier in my location and lower ping to server as well.

[arapurayil]

@jenabaivab next dns is great but most of it is achieved by blocklists. You can check: https://github.com/nextdns/metadata. If you cross check the domains in their blocklist and with another blocklist like say oisd or energized, you'll find that all of it is pretty much covered. It'd be awesome if this was acheieved programatically. DNS rebinding is important and I've implemented it for IPv4 and IPv6 private addresses. I do prefetching of cached responses also - which is what I think cache boost stands for. Do get in touch if you think of any new features. I'm always willing to improve the service.

Thanks for letting me know about webwerks, but couldn't find pricing info on their site. I'm using aws @ mumbai now. When I switched to the new server I upstreamed the requests to Quad9 to prevent any downtime, it has since then reverted to my recursive server. Can you check the ping time now? It should show an improvement.

[jenabaivab]

@arapurayil averaging 45ms which is pretty solid for my location I would say considering all server locations are away from here(Pune, Mumbai etc). Even cloudfllare gives 50 ms on a 150mbps connection.

Also, didn't you set up web werks already today once? Because I had done some leak tests around 4 AM (precisely 2 hours after you changed servers)and found my requests going to Web Werks IP addresses. But yeah, there is no pricing breakdown on their website. I'm guessing it's on a contact basis.

Additionally, isn't cloudflare the quickest resolver? I mean they also have their security focused resolver 1.1.1.2.

Do get in touch if you think of any new feature

Bit of a reach but, anonymized EDNS? Essentially privacy + quick resolving power.

[arapurayil]

@jenabaivab Even though I changed the frontend I kept the upstream server to Quad9 till ~ afternoon. I guess Quad9 must have a spot in webwerks which is why it showed up in the test results. You're abosutely right about Cloudflare. They're generally the quickest resolver and have the greatest PoPs out of all public resolvers but they're no match for a local recursor. Besides I'm trying to avoid my dependence on them as I'm wary of too much centralization.

I could provide some sort of anonymized EDNS if I had servers at different locations. But does it really make much of an impact? AFAIK there shouldn't be much of a difference if the client is from India. There is a potential issue if the client is in a far away location and is pointed to an edge server in India. Though I'm not sure about netflix, most CDNS use anycast anyway so there shouldn't be any real world difference.

[jenabaivab]

Yeah it's not a very big difference if the recursor is in India itself. Like I said, it's a bit of a reach.

Probably another out of reach wish, do you think log distinction can be achieved? Check adblocker.cloud or OpenDNS. They usually capture your IP and cross-reference the DNS logs in the system. Even if there isn't control over it, ability to monitor the requests would be huge.

Also, I really think you should decide on a name for this project. Much easier to get traction while sharing; unless you want to keep it light on the server of course and avoid a lot of traffic.

[arapurayil]

That is a neat feature to have, but it would mean compromising on the privacy aspect. I don't really want to move away from a zero-log policy with this server. My focus with this server is security. privacy and performance.

I have plans to run another DNS server with a much more proper sounding name perhaps I'll try this out there.

[jenabaivab]

A new DNS server? What are you planning on doing different there? Will it be non-filtering or something? Any tentative time of release?

[arapurayil]

All a bit nebulous, but it'll definitely filter domains and cover multiple countries. Will support all encrypted protocols. But most of the changes will be at the backend. Right now I am using unbound, nginx, proxies for the protocols, etc. I am hoping to replace all that with my own system built on top either CoreDNS or a Go DNS resolver. I also hope to have an API inplace for querying stats,logs etc.

Anyway, I'm closing this issue as I think the original issue has been addressed. Feel free to reopen if you've any other issues/queries.

[arapurayil]

@jenabaivab I'm not able to devote much time for new website. But you can checkout announcements here. I've changed the the blocklist to aBL. Test it out and pls report any issues you may find.

[jenabaivab]

@arapurayil will do. So I guess instead of OISD, this list has the sources it uses? Hopefully they get cleaned up soon with false positive reports. Will let you know if I find any.