ISSUE - Blocking (beta 5)

[iWARR]

Good news:

With new commented option # ignored_qtypes = now I can see the DS type, that was hided for me before.

Some examples from my main DNS.log :

[2018-01-20 16:44:59]   127.0.0.1   com DS
[2018-01-20 16:44:59]   127.0.0.1   com DS
[2018-01-20 16:44:59]   127.0.0.1   com DS
...
[2018-01-20 16:44:59]   127.0.0.1   greasyfork.org  DS
[2018-01-20 16:44:59]   127.0.0.1   greasyfork.org  DS
[2018-01-20 16:44:59]   127.0.0.1   greasyfork.org  DS
...

Also I see AAAA, A, TLSA, DNSKEY Then I will choose, what I want to exclude from my main DNS.log. Thanks!

Bad news - almost same ISSUE with Blocking:

Blocking still not working for me as expected. This time some another one rule was suddenly choosen from the middle of my blocking (300+ rules) list.

After service restart/clearing logs/clering dns cache/clearing browser cache, exactly only this (one, same, suddenly chosen) rule appears again in logs...

[iWARR]

That was my report for you about Beta 4, but anyway, I see it still urgent:

I've prepeared small real test for you to reproduce.

1) Lets use site for testing: https://technet.microsoft.com

2) Lets put in the blocklist two rules, for example:

*social.microsoft*
*vortex.data.*

(Also, I still keep my other rules active, below this two, if any. It's not significant for the proof of this test, but additional info for me)

3) Run testing web-page in your Browser

Clear all logs, then retest and check your blocking log Use commands to restart several times and test:

dnscrypt-proxy -service restart ipconfig /flushdns

---

My results (Beta 4):

• First time *vortex.data.* does worked, *social.microsoft* - doesn't. • Then nothing was appeared in my log at all. • Then *vortex.data.* (only) does worked again • *social.microsoft* - doesn't blocking at all (never appeared in log), however I see i2.services.social.microsoft.com in my DNS.log • Swapping by places this two rules (1/2 , 2/1) have no luck.

Something stuck with blocking for me...

[jedisct1]

It's so much better with an example!

To summarize:

Blocking example.com used to block example.com (exact match) as well as ample.com (a substring), but not xxxexample.comxxx as intended.

This has been fixed. Gonna tag a new beta, that has new features for you to play with (nx_log and prefixes).

[iWARR]

Wow! Super!!!!

[iWARR]

You are so fast with releases. Speedy : )

Until I didn't installed beta 6, I want take your attention on some blocking algo nuances. Of course, with examples, as usual.

Rule:

ad.*

What I'm expecting:

ad.example.com..............................Will be blocked. OK mad.corporations.com.................Will NOT be bloked

Why?

• Because ad.* not the same as *ad.* • Because ad.example.com = http(s)://ad.example.com = www.ad.example.com

But not the same as:

mad.corporations.com....................Will NOT be bloked host.ad.example.com.......................Will NOT be blocked

So, the algorithm for all rules must understand, if the choosen domain name START from the beginning or NOT START from the beginning:

ad.example-1.com..............................from the beginning (blocked) ad.host.example-2. com...................from the beginning (blocked)

mad.corporations.com....................not from the beginning (not blocked) host.ad.example.com........................not from the beginning (not blocked)

This nuance very important to keep the rules flexible, correct and deeply selective.

UPD: forgot about = https://

[jedisct1]

That's how it works.

http:// and https:// are unrelated to DNS.

[iWARR]

OMG. Learn through the ages, and die stupid. This is my sad story :)

[iWARR]

• Well, I can say ....vortex.data... link is not always appears as active connection on our test website in some case (while ...social.microsoft.com link stays frequently active). I can check it by uBlock Origin in the Browser.

• Yesterday I downgraded to v1.9.5 again. For now, blocking logs looks different compared to v2 and much, much more crowded. I can say, sometimes v1 also may install with "half-stuck" and have a bit imperfect blocking log (I remember such situations). In this case I need reinstall service several times with rebooting and don't touch the log files (don't delete/empty logs files until dnscrypt-proxy will start working as expected). When everything looking good, later I can safely empty logs inside the file, time to time.

Seems like I have to act with v2 in the same way until it will be fixed... It takes much more time to switch/ /setup/N-times reboot/test, but it is the only way I see...

May be this behaviour will suggest you a bit, where the problem is.

[jedisct1]

Can you provide examples of rules/query pairs that trigger a block with v1 and not with v2?

[iWARR]

I'm a bit confused about what I can propose... Because blocking:

• in the v1, basicly, working - wide list (many different blocked domains) • in the v2, basicly, not working - thin list with ~2/3/4 active rules in the log (random from beta to beta)

Hm-m-m... For example I never seen in v2 blocking log such well known domain like:

www.google-analytics.com - appears not often, but regular (Rule: *google-analytic*) www.googletagmanager.com - appears not often, (Rule: *googletag*) apis.google.com - (Rule: *apis.google.*) adservice.google.com.ua - (Rule: *adserv*) time.windows.com - I'm using other server for the OS time setting (Rule: *time.windows.*) af.opera.com - appears 100% - unstoppable queries flow, as I'm using "Opera" (Rule: *af.opera.*)

All this domains NEVER appeared in any v2 beta.

[jedisct1]

Unfortunately I can't reproduce this. It would help to have a complete rules file, and individual queries.

Beta7 is out, and includes a built-in DNS client tool.

You can use it like this:

dnscrypt-proxy --resolve apis.google.com

Please use this instead of a web browser.

Now that I think about it, that client should accept a server IP instead of using the system configuration. That will be for a next one. Still, that will be better than a web browser.

Also, use only one wildcard if possible. time.windows.* and *.google-analytics.com are way faster than *xxx*.

[iWARR]

What I had in the last beta 6:

[2018-01-21 00:10:07] 127.0.0.1 com *live-com* [2018-01-21 00:15:41] 127.0.0.1 _ *nexusrules* [2018-01-21 00:48:58] 127.0.0.1 net *adnet.* [2018-01-21 01:09:58] 127.0.0.1 _ *quantserve.*

_ - local geo-regions hided here by me

That it... Only 4 rules worked (many entries for the each one)... And not informative, what domains were blocked, yeah?

[jedisct1]

Please provide:

• A complete blacklist.txt file.
• Names for which dnscrypt-proxy --resolve <name> returns a valid response instead of being blocked as expected.

[iWARR]

You are right about speed, but you are wrong about flexibility of rules (range). There are many other full domain names will be blocked with universal rules.

Yes, you have to be ensure that domain don't have any pefixes at the beginning to use only one wildcard "if possible"

For example:

Rule: *google-analytic* should block:

www.google-analytics.com www-google-analytics.l.google.com ssl-google-analytics.l.google.com ...and many others.

And existing variants of google-analytic inside the name (without s at the end). Did you know it?

*lytics* - may block hudreds of analytics domains in different variations from tens different vendors.

Please provide

When I'll install and test the new version, I will. Right?

[iWARR]

Please provide:

Later, can I send it to you in some secure way, but not here (in the puplic)? May be as private message or smth? Send to me the message in private, i'll reply to you. Ok?

[jedisct1]

It's a shot in the dark, but remembering an old (well, when fpst was introduced in version 1, so something like 2 years ago) issue, this may be due to an unoptimized ruleset with overlaps, such as having both:

*.example.com and *.abc.example.com

in the blacklist.

In that situation bc.example.com is not blocked, because the closest suffix is *.abc.example.com, and it's not at a label boundary.

But it should be blocked due to *.example.com. In fact *.abc.example.com is useless, since it's a subset.

Beta8 is out and has a fix for this.

[iWARR]

It's Alive! (C)

• Beta 8 works! I have normal logs, including blocking logs! • All log-files was created automatically (nx.log & ip-blocked.log - first time for me)

dnscrypt-proxy --resolve <name> - really, very useful

I have another good news. Some domains from the blocking rules file that was not presented in any logs until now (probably didn't have connection yet) will immidiately blocked after --resolve checking. Also they appears in the both logs. Nice!

Thank you for your great endurance and amazing skills!

[iWARR]

Now we want HELP info about some new options :)

[iWARR]

And I want give you my blocking list (if you still want it) in some secure way. To understand my logic better. But I see that GitHub doesn't have any kind of built-in private messagser or smth...

[jedisct1]

Now I need to stop fiddling with this, and catch up with my real work, because unfortunately, dnscrypt-proxy doesn't pay the bills.

Thanks for all your bug reports, and for the good news!