DNSCrypt / dnscrypt-proxy

dnscrypt-proxy 2 - A flexible DNS proxy, with support for encrypted DNS protocols.
https://dnscrypt.info
ISC License
11.44k stars 1.01k forks source link

[mastad0n's list] Remote source list is ignored: No servers configured. #333

Closed drew1kun closed 6 years ago

drew1kun commented 6 years ago

Running dnscrypt-proxy2 on raspbian stretch based pihole. Using dnscrypt-proxy-linux_arm-2.0.8.tar.gz.

I am trying to use country based source list, say I want today to use only servers in Canada.

I commented the whole [sources.'public-resolvers'] section.

Added [sources.'canada'] section:

  [sources.'canada']
  urls = ['https://mastad0n.github.io/resolvers/v2/canada.md']
  cache_file ='canada.md'
  minisign_key = 'RWQ2ITcpSLrSUZKNN1WkheeG7A6vVkBSPbAUKQrTismFrpTjBuFELeZb'
  refresh_delay = 72
  prefix = ''

Set log_level = 6 Just trying to see what's going on...

Trying to reinstall the service:

# dnscrypt-proxy -service uninstall
[2018-04-08 17:35:59] [NOTICE] dnscrypt-proxy 2.0.8
[2018-04-08 17:35:59] [FATAL] Failed to uninstall DNSCrypt client proxy: "systemctl" failed: exit status 1

Then trying just stop and start again:

# dnscrypt-proxy -service stop
[2018-04-08 17:42:13] [NOTICE] dnscrypt-proxy 2.0.8
[2018-04-08 17:42:13] [FATAL] Failed to stop DNSCrypt client proxy: "systemctl" failed: exit status 5
# dnscrypt-proxy -service start
# echo $?
255

Log (btw, not what I have expected for lever 6):

[2018-04-08 17:36:09] [FATAL] No servers configured
[2018-04-08 17:36:19] [FATAL] No servers configured
[2018-04-08 17:41:14] [FATAL] No servers configured

But if I add the [static.'ca-1'] static server record(from that same canada.md list btw) to [static] section, then it works as expected ( and I know for sure that this server matches all require_* filters)

  [static.'ca-1']
  #dnscrypt.ca-1
  #Uncensored DNSSEC validating and log-free
  stamp = 'sdns://AQMAAAAAAAAAFDE5OS4xNjcuMTMwLjExODo1MzUzIHT3RVUXvCb3EXflbXKTJ4hscpFbP0YoMD-RDEfDjoJ5HTIuZG5zY3J5cHQtY2VydC5kbnNjcnlwdC5jYS0x'

Why does it force me to use static record, while it has the remote source list record in configuration?? Isn't it supposed to use the fastest server from that list matching all require_* filters? The documentation states that [static] section is Optional. And why the log_level = 6 returns the same amount of data as the log level = 2 - just one line?

One more interesting thing:

# dnscrypt-proxy -list
ca-1

# dnscrypt-proxy -list-all
dnscrypt.ca-1
dnscrypt.ca-2
dnscrypt.ca-3
cs-caeast
cs-cawest
ca-1

if I add another server record (again from that same list) and repeat:

# dnscrypt-proxy -list
ca-1
cs-cawest

So it sees the remote list (output of -list-all command) but matches required_* only if those same servers are in [static] list, which contradicts the documentation.

And one thing more... No matter which server I specify in [static] and no matter what dnscrypt-proxy -list shows I still seem to be using the same resolver ip

(I am doing dnscrypt-proxy -service stop && dnscrypt-proxy -service uninstall && dnscrypt-proxy -service install && dnscrypt-proxy -service start every time I make changes in config file):

# dnscrypt-proxy -list
ca-1
# dnscrypt-proxy -resolve google.com | grep Resolver
Resolver IP:    162.221.207.228

# dnscrypt-proxy -list
cs-cawest
# dnscrypt-proxy -resolve google.com | grep Resolver
Resolver IP:    162.221.207.228

# dnscrypt-proxy -list
cs-cawest
# dnscrypt-proxy -resolve google.com | grep Resolver
Resolver IP:    162.221.207.228

At this point I am so confused that I no longer know which server is being used...

drew1kun commented 6 years ago

My dnscrypt-proxy.toml:

#=========================== Global settings ==============================
## List of servers to use
## If this line is commented, all registered servers matching the require_* filters will be used
## The proxy will automatically pick the fastest, working servers from the list.
## Remove the leading # first to enable this; lines starting with # are ignored.
# server_names = ['scaleway-fr', 'google', 'yandex']
#server_names = ['ca-1']

## List of local addresses and ports to listen to. Can be IPv4 and/or IPv6.
## Note: When using systemd socket activation, choose an empty set (i.e. [] ).
listen_addresses = ['127.0.0.1:41', '[::1]:41']

## Maximum number of simultaneous client connections to accept
max_clients = 250

## Require servers (from static + remote sources) to satisfy specific properties
# Use servers reachable over IPv4
ipv4_servers = true

# Use servers reachable over IPv6 -- Do not enable if you don't have IPv6 connectivity
ipv6_servers = false

# Use servers implementing the DNSCrypt protocol
dnscrypt_servers = true

# Use servers implementing the DNS-over-HTTPS protocol
doh_servers = true

## Require servers defined by remote sources to satisfy specific properties
# Server must support DNS security extensions (DNSSEC)
require_dnssec = true

# Server must not log user queries (declarative)
#require_nolog = true

# Server must not enforce its own blacklist (for parental control, ads blocking...)
require_nofilter = true

## Always use TCP to connect to upstream servers
force_tcp = false

## How long a DNS query will wait for a response, in milliseconds
timeout = 2500

## Load-balancing strategy: 'p2' (default), 'ph', 'fastest' or 'random'
# lb_strategy = 'p2'

## Log level (0-6, default: 2 - 0 is very verbose, 6 only contains fatal errors)
log_level = 6

## log file for the application
log_file = 'dnscrypt-proxy.log'

## Use the system logger (syslog on Unix, Event Log on Windows)
#use_syslog = true

## Delay, in minutes, after which certificates are reloaded
cert_refresh_delay = 240

## Fallback resolver
## This is a normal, non-encrypted DNS resolver, that will be only used
## for one-shot queries when retrieving the initial resolvers list, and
## only if the system DNS configuration doesn't work.
## No user application queries will ever be leaked through this resolver,
## and it will not be used after IP addresses of resolvers URLs have been found.
## It will never be used if lists have already been cached, and if stamps
## don't include host names without IP addresses.
## It will not be used if the configured system DNS works.
## A resolver supporting DNSSEC is recommended. This may become mandatory.
##
## People in China may need to use 114.114.114.114:53 here.
fallback_resolver = '9.9.9.9:53'

## Never try to use the system DNS settings; unconditionally use the
## fallback resolver.
ignore_system_dns = false

## Automatic log files rotation
# Maximum log files size in MB
log_files_max_size = 10

# How long to keep backup files, in days
log_files_max_age = 7

# Maximum log files backups to keep (or 0 to keep all backups)
log_files_max_backups = 1
#==========================================================================

#============================== Filters ===================================
## Immediately respond to IPv6-related queries with an empty response
## This makes things faster when there is no IPv6 connectivity, but can
## also cause reliability issues with some stub resolvers. In
## particular, enabling this on macOS is not recommended.
block_ipv6 = false
#==========================================================================

#==== Route queries for specific domains to a dedicated set of servers ====
## Example map entries (one entry per line):
## example.com 9.9.9.9
## example.net 9.9.9.9,8.8.8.8
# forwarding_rules = 'forwarding-rules.txt'
#==========================================================================

#============================= Cloaking rules =============================
## Cloaking returns a predefined address for a specific name.
## In addition to acting as a HOSTS file, it can also return the IP address
## of a different name. It will also do CNAME flattening.
##
## Example map entries (one entry per line)
## example.com     10.1.1.1
## www.google.com  forcesafesearch.google.com
# cloaking_rules = 'cloaking-rules.txt'
#==========================================================================

#=============================== DNS cache ================================
## Enable a DNS cache to reduce latency and outgoing traffic
cache = true

## Cache size
cache_size = 256

## Minimum TTL for cached entries
cache_min_ttl = 600

## Maximum TTL for cached entries
cache_max_ttl = 86400

## TTL for negatively cached entries
cache_neg_ttl = 60
#==========================================================================

#============================ Query logging ===============================
## Log client queries to a file
[query_log]
  ## Path to the query log file (absolute, or relative to the same directory as the executable file)
  # file = 'query.log'

  ## Query log format (currently supported: tsv and ltsv)
  format = 'tsv'

  ## Do not log these query types, to reduce verbosity. Keep empty to log everything.
  # ignored_qtypes = ['DNSKEY', 'NS']
#==========================================================================

#======================== Suspicious queries logging ======================
## Log queries for nonexistent zones
## These queries can reveal the presence of malware, broken/obsolete applications,
## and devices signaling their presence to 3rd parties.
[nx_log]
  ## Path to the query log file (absolute, or relative to the same directory as the executable file)
  # file = 'nx.log'

  ## Query log format (currently supported: tsv and ltsv)
  format = 'tsv'
#==========================================================================

#================== Pattern-based blocking (blacklists) ===================
## Blacklists are made of one pattern per line. Example of valid patterns:
##
##   example.com
##   *sex*
##   ads.*
##   ads*.example.*
##   ads*.example[0-9]*.com
##
## Example blacklist files can be found at https://download.dnscrypt.info/blacklists/
## A script to build blacklists from public feeds can be found in the
## `utils/generate-domains-blacklists` directory of the dnscrypt-proxy source code.
[blacklist]
  ## Path to the file of blocking rules (absolute, or relative to the same directory as the executable file)
  # blacklist_file = 'blacklist.txt'

  ## Optional path to a file logging blocked queries
  # log_file = 'blocked.log'

  ## Optional log format: tsv or ltsv (default: tsv)
  # log_format = 'tsv'
#==========================================================================

#==== Pattern-based IP blocking (IP blacklists) =====
## IP blacklists are made of one pattern per line. Example of valid patterns:
##
##   127.*
##   fe80:abcd:*
##   192.168.1.4
[ip_blacklist]
  ## Path to the file of blocking rules (absolute, or relative to the same directory as the executable file)
  # blacklist_file = 'ip-blacklist.txt'

  ## Optional path to a file logging blocked queries
  # log_file = 'ip-blocked.log'

  ## Optional log format: tsv or ltsv (default: tsv)
  # log_format = 'tsv'
#==========================================================================

#====================== Time access restrictions ==========================
## One or more weekly schedules can be defined here.
## Patterns in the name-based blocklist can optionally be followed with @schedule_name
## to apply the pattern 'schedule_name' only when it matches a time range of that schedule.
##
## For example, the following rule in a blacklist file:
## *.youtube.* @time-to-sleep
## would block access to Youtube only during the days, and period of the days
## define by the 'time-to-sleep' schedule.
##
## {after='21:00', before= '7:00'} matches 0:00-7:00 and 21:00-0:00
## {after= '9:00', before='18:00'} matches 9:00-18:00
[schedules]
  # [schedules.'time-to-sleep']
  # mon = [{after='21:00', before='7:00'}]
  # tue = [{after='21:00', before='7:00'}]
  # wed = [{after='21:00', before='7:00'}]
  # thu = [{after='21:00', before='7:00'}]
  # fri = [{after='23:00', before='7:00'}]
  # sat = [{after='23:00', before='7:00'}]
  # sun = [{after='21:00', before='7:00'}]

  # [schedules.'work']
  # mon = [{after='9:00', before='18:00'}]
  # tue = [{after='9:00', before='18:00'}]
  # wed = [{after='9:00', before='18:00'}]
  # thu = [{after='9:00', before='18:00'}]
  # fri = [{after='9:00', before='17:00'}]
#==========================================================================

#================================ Servers =================================
## Remote lists of available servers
## Multiple sources can be used simultaneously, but every source
## requires a dedicated cache file.
##
## Refer to the documentation for URLs of public sources.
##
## A prefix can be prepended to server names in order to
## avoid collisions if different sources share the same for
## different servers. In that case, names listed in `server_names`
## must include the prefixes.
##
## If the `urls` property is missing, cache files and valid signatures
## must be already present; This doesn't prevent these cache files from
## expiring after `refresh_delay` hours.
[sources]
  ## An example of a remote source
#  [sources.'public-resolvers']
#  urls = ['https://raw.githubusercontent.com/DNSCrypt/dnscrypt-resolvers/master/v2/public-resolvers.md', 'https://download.dnscrypt.info/resolvers-list/v2/public-resolvers.md']
#  cache_file = 'public-resolvers.md'
#  minisign_key = 'RWQf6LRCGA9i53mlYecO4IzT51TGPpvWucNSCh1CBM0QTaLn73Y7GFO3'
#  refresh_delay = 72
#  prefix = ''

#  [sources.'iceland']
#  urls = ['https://mastad0n.github.io/resolvers/v2/iceland.md']
#  cache_file ='iceland.md'
#  minisign_key = 'RWQ2ITcpSLrSUZKNN1WkheeG7A6vVkBSPbAUKQrTismFrpTjBuFELeZb'
#  refresh_delay = 72
#  prefix = ''

  [sources.'canada']
  urls = ['https://mastad0n.github.io/resolvers/v2/canada.md']
  cache_file ='canada.md'
  minisign_key = 'RWQ2ITcpSLrSUZKNN1WkheeG7A6vVkBSPbAUKQrTismFrpTjBuFELeZb'
  refresh_delay = 72
  prefix = ''

  ## Another example source, with resolvers censoring some websites not appropriate for children
  ## This is a subset of the `public-resolvers` list, so enabling both is useless
  #  [sources.'parental-control']
  #  urls = ['https://raw.githubusercontent.com/DNSCrypt/dnscrypt-resolvers/master/v2/parental-control.md', 'https://download.dnscrypt.info/resolvers-list/v2/parental-control.md']
  #  cache_file = 'parental-control.md'
  #  minisign_key = 'RWQf6LRCGA9i53mlYecO4IzT51TGPpvWucNSCh1CBM0QTaLn73Y7GFO3'

## Optional, local, static list of additional servers
## Mostly useful for testing your own servers.
[static]
  #[static.'ca-1']
  # dnscrypt.ca-1
  #Uncensored DNSSEC validating and log-free
  #stamp = 'sdns://AQMAAAAAAAAAFDE5OS4xNjcuMTMwLjExODo1MzUzIHT3RVUXvCb3EXflbXKTJ4hscpFbP0YoMD-RDEfDjoJ5HTIuZG5zY3J5cHQtY2VydC5kbnNjcnlwdC5jYS0x'

  # [static.'google']
  # stamp = 'sdns://AgUAAAAAAAAAACDyXGrcc5eNecJ8nomJCJ-q6eCLTEn6bHic0hWGUwYQaA5kbnMuZ29vZ2xlLmNvbQ0vZXhwZXJpbWVudGFs'
jedisct1 commented 6 years ago

Hi,

Unfortunately, @mastad0n 's list is not up to date, and the stamps are not correct. In particular, it doesn't have correct attributes for the filters.

Maintaining these lists is time consuming, and maybe he doesn't have much time to do so.

Maybe you could maintain similar lists?

drew1kun commented 6 years ago

Oh I see... So the list does not work because the minisign is not correct, but the stamps for idividual server instances are correct, that is why when I add them to [static] all works. Do I get it right?

But why Resolver IP is always 162.221.207.228 (which is not a fallback, as I understand, because the dnscrypt-proxy even with log_level = 6 does not say anything like "using fallback server")? How can I check what is 162.221.207.228?

First I thought that this may be 9.9.9.9's anycast or some canadaian server.

# whois 162.221.207.228
% IANA WHOIS server
% for more information on IANA, visit http://www.iana.org
% This query returned 1 object

refer:        whois.arin.net

inetnum:      162.0.0.0 - 162.255.255.255
organisation: Administered by ARIN
status:       LEGACY

whois:        whois.arin.net

changed:      1993-05
source:       IANA

#
# ARIN WHOIS data and services are subject to the Terms of Use
# available at: https://www.arin.net/whois_tou.html
#
# If you see inaccuracies in the results, please report at
# https://www.arin.net/public/whoisinaccuracy/index.xhtml
#

#
# Query terms are ambiguous.  The query is assumed to be:
#     "n + 162.221.207.228"
#
# Use "?" to get help.
#

#
# The following results may also be obtained via:
# https://whois.arin.net/rest/nets;q=162.221.207.228?showDetails=true&showARIN=false&showNonArinTopLevelNet=false&ext=netref2
#

NetRange:       162.221.200.0 - 162.221.207.255
CIDR:           162.221.200.0/21
NetName:        ESD-UNITED-V4
NetHandle:      NET-162-221-200-0-1
Parent:         NET162 (NET-162-0-0-0-0)
NetType:        Direct Allocation
OriginAS:       AS11831
Organization:   eSecureData (ESECU-4)
RegDate:        2013-09-20
Updated:        2013-09-21
Ref:            https://whois.arin.net/rest/net/NET-162-221-200-0-1

OrgName:        eSecureData
OrgId:          ESECU-4
Address:        1478 Hartley Ave.
City:           Coquitlam
StateProv:      BC
PostalCode:     V3K 7A1
Country:        CA
RegDate:        2008-03-31
Updated:        2017-01-28
Ref:            https://whois.arin.net/rest/org/ESECU-4

OrgNOCHandle: SUPPO579-ARIN
OrgNOCName:   Support Department
OrgNOCPhone:  +1-800-620-1985
OrgNOCEmail:  support@esecuredata.com
OrgNOCRef:    https://whois.arin.net/rest/poc/SUPPO579-ARIN

OrgTechHandle: SUPPO579-ARIN
OrgTechName:   Support Department
OrgTechPhone:  +1-800-620-1985
OrgTechEmail:  support@esecuredata.com
OrgTechRef:    https://whois.arin.net/rest/poc/SUPPO579-ARIN

OrgAbuseHandle: SUPPO579-ARIN
OrgAbuseName:   Support Department
OrgAbusePhone:  +1-800-620-1985
OrgAbuseEmail:  support@esecuredata.com
OrgAbuseRef:    https://whois.arin.net/rest/poc/SUPPO579-ARIN

#
# ARIN WHOIS data and services are subject to the Terms of Use
# available at: https://www.arin.net/whois_tou.html
#
# If you see inaccuracies in the results, please report at
# https://www.arin.net/public/whoisinaccuracy/index.xhtml
#

But it is probably not a fallback's anycast, because when I completely remove the canada's server static records and add Iceland, the resolver changes:

dnscrypt-proxy -resolve google.com | grep Resolver
Resolver IP:    93.95.226.165 (vps-93-95-226-165.1984.is.)

Although when I use another iceland server, then resolution does not work, which proves that it tries to use another server. But why it does not use fallback if the server does not work????

jedisct1 commented 6 years ago

162.221.207.228 is the cs-cawest resolver, operated by cryptostorm.is.

An organization in Iceland but a server in Canada :)

drew1kun commented 6 years ago

I know, but it does not answer my questions..

I would like to try to maintain the list, but I have no idea where to get any information about those dns servers...

jedisct1 commented 6 years ago

Most, if not all of them, should have some information about their location in their description.

jedisct1 commented 6 years ago

This is the list as a raw file: https://download.dnscrypt.info/dnscrypt-resolvers/v2/public-resolvers.md

Or rendered: https://github.com/DNSCrypt/dnscrypt-resolvers/blob/master/v2/public-resolvers.md

The location of the resolvers is visible either in the name (cpunks-ru) or in the description (Datacenter in Germany).

The location is also visible on the map, or here: https://download.dnscrypt.info/dnscrypt-resolvers/json/public-resolvers.json

drew1kun commented 6 years ago

Oh so you mean that I could use servers only from public-resolvers.md and just re-arrange them in a country-based lists? I definitely can do that..

jedisct1 commented 6 years ago

Yes, exactly :)

I can try to make something semi-automated, but there will be errors, especially with anycast servers.

drew1kun commented 6 years ago

Sounds good! But where do you get all that information about the servers for public-resolvers.md?

jedisct1 commented 6 years ago

What information are you referring to? The descriptions?

drew1kun commented 6 years ago

The fact of server existence, description(dnssec validation, logging), stamp (what is it btw), the ip address (for testing, so you know for sure that proxy is using the server it supposed to use..

ghost commented 6 years ago

Hi everyone, I am very sorry, due to personal reasons I cannot maintain these lists anymore.

@jedisct1 thank you so much for all your work

ghost commented 6 years ago

https://filetransfer.io/data-package/XdcrogCa

jedisct1 commented 6 years ago

Thanks a lot for having built and maintained these lists, as well as all your help testing dnscrypt-proxy.

Having per-country was a fantastic idea, and I hope someone will be able to pursue the work you started.

Cheers!