DNSCrypt / dnscrypt-proxy

dnscrypt-proxy 2 - A flexible DNS proxy, with support for encrypted DNS protocols.
https://dnscrypt.info
ISC License
11.45k stars 1.01k forks source link

2.0.9 beta 2 available for testing #335

Closed jedisct1 closed 6 years ago

jedisct1 commented 6 years ago

New in beta 2:

In beta 1:

jedisct1 commented 6 years ago

Here's a little trick: you can whitelist things that are not blacklisted and use that to log only some queries.

Ssince whitelists can have their own log file, you can log queries matching specific pattern in a different place than the regular query log.

Or only log queries matching some patterns, without blocking them and not log anything else.

Biggizen commented 6 years ago

After having trouble with beta1 (mentioned in issue #323), I fortunately can now give the feedback, that beta2 works. :)

zenofile commented 6 years ago

I had latency issues with 2.0.8 on a Raspberry Pi 2b and Cloudflare DoH which now seem to be solved using TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305 and keepalive = 0.

jedisct1 commented 6 years ago

@zenofx ECDHE_RSA is about 3 times faster than ECDHE_ECDSA on Raspberry Pi Zero. It should be similar on a Pi 2b. Keep keepalive to some reasonable value.

zenofile commented 6 years ago

@jedisct1 Thank you, I switched to ECDHE_RSA. Why is keepalive = 0 unreasonable though? Shouldn't this keep the connection established until the upstream server decides to terminate it?

IdleConnTimeout is the maximum amount of time an idle (keep-alive) connection will remain idle before closing itself. Zero means no limit.

ghost commented 6 years ago

dnscrypt-proxy-win64-2.0.9b2.zip / Windows 7 64-BIT : running flawlessly

Main novelty here : tls_cipher_suite = [52392, 49199] (RSA_WITH_CHACHA20_POLY1305, AES_128_GCM_SHA256), used now with beta2.

Here's a little trick: you can whitelist things that are not blacklisted and use that to log only some queries

Nice, thanks @jedisct1 ... I have the feeling I'm going to use that more than once :=)

jedisct1 commented 6 years ago

@zenofx This assumes that TCP connections are gracefully terminated.

The reality is different, especially for a proxy that has to run in an environment where devices frequently switch IP addresses, switch network devices (Wifi / radio), turn VPNs on and off and have not so reliable WAN connections.

In these environments, connections are not terminated. They stall. The clients waits for something, but nothing comes in. Is the peer dead? The connection down? Or everything is fine but the peer just does't have anything to send? There's no way to know, except guess that something's wrong after some timeout, and force reset the connection.

Until this happens, the proxy is not responsive. DNS queries fail. People get upset. They start smashing their computer with hammers. This is not good, if only for the environment.

Having a small timeout client-side prevents this. But if you don't have a hammer, maybe you don't need this.

zenofile commented 6 years ago

@jedisct1 Maybe it would be possible to protect some of those innocent computers from hammers with HTTP2 PING frames as described in the RFC, when using a zero or high keepalive value, periodically (?) checking if an idle connection is stale or still good. This would allow to reuse the connection and save some CPU cycles, reducing latency I suppose.

The PING frame (type=0x6) is a mechanism for measuring a minimal round-trip time from the sender, as well as determining whether an idle connection is still functional. PING frames can be sent from any endpoint.

jedisct1 commented 6 years ago

That could be an option. But constant traffic on non-unlimited mobile plans would result in people smashing their phones with baseball bats when they get the bill. Not good either.

jedisct1 commented 6 years ago

(but more seriously, it looks like this is supported in Golang's net/http2, so this is doable, provided that all the servers also implement this).

zenofile commented 6 years ago

Introducing a new healthcheck option of some kind which defaults to off would fix that.

jedisct1 commented 6 years ago

That's a good suggestion. TCP keepalive can also help. But let's keep this thread about testing the beta version, so that 2.0.9 can be released as soon as it looks good enough for most people.

frederickding commented 6 years ago

Adding a data point here (running on mips64 EdgeRouter Lite):

After upgrading from 2.0.8 to 2.0.9b2 with the added config for cipher suite, CPU utilization significantly dropped. Whereas the executable used to take up >1 core at load (i.e., while running a DNS test like https://www.grc.com/dns/dns.htm), the change in cipher suites keeps it at relatively low CPU -- rarely ramping up above 15%.

This makes it feasible to run long-term as part of a router appliance. Good work!

ysz223 commented 6 years ago
  1. When two versions of 2.0.9b were run on Debian, there was a problem of disconnecting the network and restarting the computer before connecting.

  2. I have two computers, all installed with the same version of Debian 9, and the first one was successfully configured through the previous method. However, the second new computer, whether it is using the previous method or the latest method configuration, cannot be configured successfully. Can you improve the configuration of the Debian system?

  3. Every update must be uninstalled and reinstalled. Is there a way to enable automatic updates?

Thank you!

jedisct1 commented 6 years ago

Hi @ysz223,

The easiest way to switch to a new version is to simply delete the dnscrypt-proxy executable (no need to stop it first), copy the new one to the same location, then type ./dnscrypt-proxy -service restart. You will have virtually no service interruption.

Software-specific automatic updates tend to conflict with the standard package system of the distribution. So what we would really need is someone maintaining proper Debian packages.

Maybe there are Debian packages for dnscrypt-proxy 2 already?

ghost commented 6 years ago

The easiest way to switch to a new version is to simply delete the dnscrypt-proxy executable (no need to stop it first), copy the new one to the same location, then type ./dnscrypt-proxy -service restart.

What about the dnscrypt-proxy.toml configuration file? When modified by a new version, needs as well to be copied and previous user-modified settings reset. Not to mention service-install.bat, service-restart.bat, service-uninstall.bat when modified...

I'm not correcting the developer! Rather wondering what I could be incorrectly understanding. Of course I'm assuming Debian packages are set as those specific to Windows, which might not be true.

ysz223 commented 6 years ago

@jedisct1 My software packages are downloaded and run independently on the site and do not use Debian distributions.

JasonSwindle commented 6 years ago

I've been testing on an rPi2, and it has been solid. One thing I notices was, when syslog support is enabled, only the daemon logs show up in syslog but not query logs. Bug or by design?


##############################################
#                                            #
#        dnscrypt-proxy configuration        #
#                                            #
##############################################

## This is an example configuration file.
## You should adjust it to your needs, and save it as "dnscrypt-proxy.toml"
##
## Online documentation is available here: https://dnscrypt.info/doc

##################################
#         Global settings        #
##################################

## List of servers to use
## If this line is commented, all registered servers matching the require_* filters
## will be used
## The proxy will automatically pick the fastest, working servers from the list.
## Remove the leading # first to enable this; lines starting with # are ignored.

# server_names = ['scaleway-fr', 'google', 'yandex']
server_names = ['cloudflare', 'google']

## List of local addresses and ports to listen to. Can be IPv4 and/or IPv6.
## Note: When using systemd socket activation, choose an empty set (i.e. [] ).

listen_addresses = ['192.168.1.107:53']

## Maximum number of simultaneous client connections to accept

max_clients = 250

## Require servers (from static + remote sources) to satisfy specific properties

# Use servers reachable over IPv4
ipv4_servers = true

# Use servers reachable over IPv6 -- Do not enable if you don't have IPv6 connectivity
ipv6_servers = false

# Use servers implementing the DNSCrypt protocol
dnscrypt_servers = true

# Use servers implementing the DNS-over-HTTPS protocol
doh_servers = true

## Require servers defined by remote sources to satisfy specific properties

# Server must support DNS security extensions (DNSSEC)
require_dnssec = true

# Server must not log user queries (declarative)
require_nolog = true

# Server must not enforce its own blacklist (for parental control, ads blocking...)
require_nofilter = true

## Always use TCP to connect to upstream servers

force_tcp = false

## How long a DNS query will wait for a response, in milliseconds

timeout = 2500

## Load-balancing strategy: 'p2' (default), 'ph', 'fastest' or 'random'

# lb_strategy = 'p2'

## Log level (0-6, default: 2 - 0 is very verbose, 6 only contains fatal errors)

log_level = 0

## log file for the application

#log_file = '/var/log/dnscrypt-proxy/dnscrypt-proxy.log'

## Use the system logger (syslog on Unix, Event Log on Windows)

use_syslog = true

## Delay, in minutes, after which certificates are reloaded

cert_refresh_delay = 240

## Fallback resolver
## This is a normal, non-encrypted DNS resolver, that will be only used
## for one-shot queries when retrieving the initial resolvers list, and
## only if the system DNS configuration doesn't work.
## No user application queries will ever be leaked through this resolver,
## and it will not be used after IP addresses of resolvers URLs have been found.
## It will never be used if lists have already been cached, and if stamps
## don't include host names without IP addresses.
## It will not be used if the configured system DNS works.
## A resolver supporting DNSSEC is recommended. This may become mandatory.
##
## People in China may need to use 114.114.114.114:53 here.

fallback_resolver = '9.9.9.9:53'

## Never try to use the system DNS settings; unconditionally use the
## fallback resolver.

ignore_system_dns = true

## Automatic log files rotation

# Maximum log files size in MB
log_files_max_size = 5

# How long to keep backup files, in days
log_files_max_age = 7

# Maximum log files backups to keep (or 0 to keep all backups)
log_files_max_backups = 1

#########################
#        Filters        #
#########################

## Immediately respond to IPv6-related queries with an empty response
## This makes things faster when there is no IPv6 connectivity, but can
## also cause reliability issues with some stub resolvers. In
## particular, enabling this on macOS is not recommended.

block_ipv6 = false

##################################################################################
#        Route queries for specific domains to a dedicated set of servers        #
##################################################################################

## Example map entries (one entry per line):
## example.com 9.9.9.9
## example.net 9.9.9.9,8.8.8.8

# forwarding_rules = 'forwarding-rules.txt'

###############################
#        Cloaking rules       #
###############################

## Cloaking returns a predefined address for a specific name.
## In addition to acting as a HOSTS file, it can also return the IP address
## of a different name. It will also do CNAME flattening.
##
## Example map entries (one entry per line)
## example.com     10.1.1.1
## www.google.com  forcesafesearch.google.com

# cloaking_rules = 'cloaking-rules.txt'

###########################
#        DNS cache        #
###########################

## Enable a DNS cache to reduce latency and outgoing traffic

cache = true

## Cache size

cache_size = 256

## Minimum TTL for cached entries

cache_min_ttl = 600

## Maximum TTL for cached entries

cache_max_ttl = 86400

## TTL for negatively cached entries

cache_neg_ttl = 60

###############################
#        Query logging        #
###############################

## Log client queries to a file

[query_log]

  ## Path to the query log file (absolute, or relative to the same directory as the executable file)

  # file = '/var/log/dnscrypt-proxy/query.log'

  ## Query log format (currently supported: tsv and ltsv)

  format = 'tsv'

  ## Do not log these query types, to reduce verbosity. Keep empty to log everything.

  # ignored_qtypes = ['DNSKEY', 'NS']

############################################
#        Suspicious queries logging        #
############################################

## Log queries for nonexistent zones
## These queries can reveal the presence of malware, broken/obsolete applications,
## and devices signaling their presence to 3rd parties.

[nx_log]

  ## Path to the query log file (absolute, or relative to the same directory as the executable file)

  # file = 'nx.log'

  ## Query log format (currently supported: tsv and ltsv)

  format = 'tsv'

######################################################
#        Pattern-based blocking (blacklists)        #
######################################################

## Blacklists are made of one pattern per line. Example of valid patterns:
##
##   example.com
##   *sex*
##   ads.*
##   ads*.example.*
##   ads*.example[0-9]*.com
##
## Example blacklist files can be found at https://download.dnscrypt.info/blacklists/
## A script to build blacklists from public feeds can be found in the
## `utils/generate-domains-blacklists` directory of the dnscrypt-proxy source code.

[blacklist]

  ## Path to the file of blocking rules (absolute, or relative to the same directory as the executable file)

  # blacklist_file = 'blacklist.txt'

  ## Optional path to a file logging blocked queries

  # log_file = 'blocked.log'

  ## Optional log format: tsv or ltsv (default: tsv)

  # log_format = 'tsv'

###########################################################
#        Pattern-based IP blocking (IP blacklists)        #
###########################################################

## IP blacklists are made of one pattern per line. Example of valid patterns:
##
##   127.*
##   fe80:abcd:*
##   192.168.1.4

[ip_blacklist]

  ## Path to the file of blocking rules (absolute, or relative to the same directory as the executable file)

  # blacklist_file = 'ip-blacklist.txt'

  ## Optional path to a file logging blocked queries

  # log_file = 'ip-blocked.log'

  ## Optional log format: tsv or ltsv (default: tsv)

  # log_format = 'tsv'

##########################################
#        Time access restrictions        #
##########################################

## One or more weekly schedules can be defined here.
## Patterns in the name-based blocklist can optionally be followed with @schedule_name
## to apply the pattern 'schedule_name' only when it matches a time range of that schedule.
##
## For example, the following rule in a blacklist file:
## *.youtube.* @time-to-sleep
## would block access to Youtube only during the days, and period of the days
## define by the 'time-to-sleep' schedule.
##
## {after='21:00', before= '7:00'} matches 0:00-7:00 and 21:00-0:00
## {after= '9:00', before='18:00'} matches 9:00-18:00

[schedules]

  # [schedules.'time-to-sleep']
  # mon = [{after='21:00', before='7:00'}]
  # tue = [{after='21:00', before='7:00'}]
  # wed = [{after='21:00', before='7:00'}]
  # thu = [{after='21:00', before='7:00'}]
  # fri = [{after='23:00', before='7:00'}]
  # sat = [{after='23:00', before='7:00'}]
  # sun = [{after='21:00', before='7:00'}]

  # [schedules.'work']
  # mon = [{after='9:00', before='18:00'}]
  # tue = [{after='9:00', before='18:00'}]
  # wed = [{after='9:00', before='18:00'}]
  # thu = [{after='9:00', before='18:00'}]
  # fri = [{after='9:00', before='17:00'}]

#########################
#        Servers        #
#########################

## Remote lists of available servers
## Multiple sources can be used simultaneously, but every source
## requires a dedicated cache file.
##
## Refer to the documentation for URLs of public sources.
##
## A prefix can be prepended to server names in order to
## avoid collisions if different sources share the same for
## different servers. In that case, names listed in `server_names`
## must include the prefixes.
##
## If the `urls` property is missing, cache files and valid signatures
## must be already present; This doesn't prevent these cache files from
## expiring after `refresh_delay` hours.

[sources]

  ## An example of a remote source

  [sources.'public-resolvers']
  urls = ['https://raw.githubusercontent.com/DNSCrypt/dnscrypt-resolvers/master/v2/public-resolvers.md', 'https://download.dnscrypt.info/resolvers-list/v2/public-resolvers.md']
  cache_file = 'public-resolvers.md'
  minisign_key = 'RWQf6LRCGA9i53mlYecO4IzT51TGPpvWucNSCh1CBM0QTaLn73Y7GFO3'
  refresh_delay = 72
  prefix = ''

  ## Another example source, with resolvers censoring some websites not appropriate for children
  ## This is a subset of the `public-resolvers` list, so enabling both is useless

  #  [sources.'parental-control']
  #  urls = ['https://raw.githubusercontent.com/DNSCrypt/dnscrypt-resolvers/master/v2/parental-control.md', 'https://download.dnscrypt.info/resolvers-list/v2/parental-control.md']
  #  cache_file = 'parental-control.md'
  #  minisign_key = 'RWQf6LRCGA9i53mlYecO4IzT51TGPpvWucNSCh1CBM0QTaLn73Y7GFO3'

## Optional, local, static list of additional servers
## Mostly useful for testing your own servers.

[static]

  # [static.'google']
  # stamp = 'sdns://AgUAAAAAAAAAACDyXGrcc5eNecJ8nomJCJ-q6eCLTEn6bHic0hWGUwYQaA5kbnMuZ29vZ2xlLmNvbQ0vZXhwZXJpbWVudGFs'
jedisct1 commented 6 years ago

Yes, the query logs go to a dedicated file. Queries, blacklists, IP blacklist, whitelists, have their own dedicated logs to keep things organized :)

JasonSwindle commented 6 years ago

Got it. I was going to push them all into syslog and ship them off the instance to process vs having to install a shipping agent. Trying to understand my usage patterns or failures before I deploy this to my family and their houses. :D

Cheeselets commented 6 years ago

FYI, I have installed 2.0.9 beta 2 on an ASUS RT-AC56U Router. [ARM v7] (2.0.8 previously working fine) It is working BUT the option to specify 'specific cipher suite' does not work : (As in dnscrypt-proxy does not work -- no dns response) [All samples from 'working' original .toml file.] (dnscrypt-proxy -check passes the original file with no errors)

## DoH: Use a specific cipher suite instead of the server preference _## 49199 = TLS_ECDHE_RSA_WITH_AES_128_GCMSHA256 _## 49195 = TLS_ECDHE_ECDSA_WITH_AES_128_GCMSHA256 _## 52392 = TLS_ECDHE_RSA_WITH_CHACHA20POLY1305 _## 52393 = TLS_ECDHE_ECDSA_WITH_CHACHA20POLY1305 _tls_ciphersuite = [52392, 49199]

If I comment out the 'tls_cipher_suite' line it works & dns replies are obtained. 'cloudflare' is the Server being used via DoH.

Other new options as below do appear to work:

## Keepalive for HTTP (HTTPS, HTTP/2) queries, in seconds keepalive = 30 ## DNSCrypt: Create a new, unique key for every single DNS query ## This may improve privacy but can also have a significant impact on CPU usage ## Only enable if you don't have a lot of network load _dnscrypt_ephemeralkeys = true ## DoH: Disable TLS session tickets ## increases privacy but also latency - Bump keepalive up to compensate. _tls_disable_sessiontickets = true

jedisct1 commented 6 years ago

Hi @Cheeselets

Since you had a previous version installed, I suspect your public-resolvers.md file is not up to date.

Try deleting it (along with the .minisig) file and restart the proxy; the new version will be downloaded.

Cheeselets commented 6 years ago

@jedisct1 Thanks for the quick reply. I followed your advice and the new version of public-resolvers.md was downloaded.

I still get the same problem. Uncomment the 'tls_cipher_suite = [52392, 49199]' line and dnscrypt-proxy stops working. Re-comment the line, restart dnscrypt and dns is working again ???!!!

Is there any diagnostic I could run to show where the problem is ? I re-checked the config file via the -check option and no errors are flagged.

It is not a major problem BUT I assumed it should work as it is in the default config file.

For completeness the config file is attached.

dnscrypt-proxy.toml.txt

VindicatorDS commented 6 years ago

@Cheeselets, I'm using dnscrypt-proxy 2.0.9b2 with cloudflare and google with DoH also on an arm v7 router. It works fine with\without tls_cipher_suite, and other combinations of suites. A few days ago, I had the same problem you're facing because I was using a local cache for public-resolvers.md and public-resolvers.minisig.

I fixed the problem by downloading them both, as suggested by @jedisct1 . If you're saying you already downloaded them both and it's still not working, and since you have changed some lines on your dnscrypt-proxy.toml (that appear OK...), maybe you should start over with a vanilla dnscrypt-proxy.toml file and only set the server line to cloudflare an see if it works. If it does, then change the cipher suite and check it again. If it doesn't work with the default conf, then there's something wrong with the build\your environment.

If it works, then keep the cloudflare and ciphersuite lines changed, and change another option. But only one at a time, until you find the combination of configuration options that are causing the problem.

jedisct1 commented 6 years ago

@Cheeselets What @VindicatorDS said :)

You can also get (many) more logs about what's going on with:

export DEBUG=1
dnscrypt-proxy -loglevel=0
ajp2k13 commented 6 years ago

@jedisct1 example-dnscrypt-proxy.toml is empty in latest 2.09 archives?

CNMan commented 6 years ago

@ajp2k13 @jedisct1 same here

ghost commented 6 years ago

Same here with dnscrypt-proxy-win64-2.0.9.zip : example-dnscrypt-proxy.toml is a 2b file (not fully empty!)

jedisct1 commented 6 years ago

Oops :)

New files are being built and should appear soon :)

ghost commented 6 years ago

New files are being built and should appear soon :)

Like cooking when the meat was exaggeratedly underdone :) Miam-miam!

jedisct1 commented 6 years ago

Hahahaha, something like that. You're excited about the meat, you want to serve it as soon as possible, and it ends up being underdone :)

ghost commented 6 years ago

Frank, you took the words out my mouth :=) I stop, I started being off-topic!

ghost commented 6 years ago

(Breakfast?)/lunch/supper ready : dnscrypt-proxy-win64-2.0.9.zip (at least) now includes full-full dnscrypt-proxy.toml. Enjoy, and thanks to the cook chef :=)

Cheeselets commented 6 years ago

@VindicatorDS & @jedisct1, Thanks. Started with Default file and added back all the settings I had, one by one, ...... it works ????

Still cannot see what is the problem with the original file as it passes -check ??? Must have 'mangled' the working 2.0.8 config file somehow ! Happy Bunny :)