Chicken-and-egg problem: fetching a set of resolvers requires a DNS resolver

[elmakong]

change [sources."proxy v1 list from github"] to [sources."proxy-v1-list-from-github"] then restart the service fixes the problem: [CRITICAL] Unable use source [proxy v1 list from github]: [Get https://raw.githubusercontent.com/DNSCrypt/dnscrypt-resolvers/master/v1/dnscrypt-resolvers.csv: dial tcp: lookup raw.githubusercontent.com: no such host]

[iWARR]

DNSCrypt-Proxy 2 v2.0.0 beta 2, Win8.1 [x64]

1) I have the same issue since Alphas, Beta1 to present Beta2.

The fix from elmakong doesn't work for me at all. So I showing you the shot with stock settings:

2) Also I can't download freshy "dnscrypt-resolvers.csv" from anywhere "as ready file", but I can open it in the browser tab and copy-paste all lines into new file in the root DNSCrypt directory. I'm using the link from the *.toml config file. This is too slow and unhandy way.

[jedisct1]

Please do not post screenshots. They are unreadable on a phone, it's impossible to copy/paste their content, and since these are just images, their content will never appear in search results if someone is looking for the same issue.

The error means that the name cannot be resolved. Did you change your DNS settings?

Resolving the URL host requires working DNS settings. If you have configured your system to use the proxy IP, and nothing else, since the proxy hasn't started yet, you can't resolve this host name. Chicken-and-egg problem.

The only ways to address that are to:

• Either use an URL that includes an IP address instead of a host name
• Or add an option to provide the IP address, in addition to the host name
• Or give a set of non-dnscrypt servers, that will be only used to fetch the source URLs

We can also delay the update after at least one valid dnscrypt-enabled server has been found, but that wouldn't resolve the first-start issue.

[iWARR]

Please do not post screenshots.

OK! Didn't knew. This time it was just a proof for the same error as elmakong has. So it will be searchable anyway.

Chicken-and-egg problem: I've guessed about cause of it... Yeah, it's not good deal to change DNS to ISP every time (and no reason: service starts, we have errors, but it works anyway). Bad. Not beautiful.

So, • I think, your idea with IP in the link is a good idea. And...

• I will prefer have "dnscrypt-resolvers.csv" just inside of the distributive as a start point. First access goes to file from the root folder, then updating after at least one valid dnscrypt-enabled server has been found.

But I made and put this file by myself in the root folder and wrote the path in the *.toml Why we have error anyway? M-m-m.... We have two sources - URL and Absolute path to the Disk at the same time... cache_file = 'C:\Program Files\DNSCrypt\dnscrypt-resolvers.csv'

• How about my Point 2 ? Downloading "dnscrypt-resolvers.csv" as "ready file" (manually)?

[jedisct1]

Th idea is that there is no source of truth any more. Instead of having a single list, you can subscribe to any number of lists. Each list can be hosted at a different location, and be signed with their own key.

The OpenNIC organization can maintain and publish their list of DNSCrypt-enabled resolvers. If you have your own servers, you can publish them as a list. If someone wants to maintain a list of resolvers that work well for a given country, they can do it.

So, you can subscribe to the lists you want, and they will be all automatically downloaded and updated.

The CSV file you are referring to is just a cache for one specific source. These cache files avoid fetching a list every time the proxy is started.

Instead of shipping one specific cache file, that may not be from a list a user want, and that will always be out of date, here is a proposal.

A reasonable assertion is that the first time the proxy is started, the DNS settings haven't been changed to 127.0.0.1 yet. So, we can download the lists.

Later, if the proxy is restarted, and the cache is still valid, we just use the cache, as we do today. If the cache is valid, but a bit old, we quickly try to download updates. If that download fails, this is fine. We use the (slightly outdated, but probably still okay) cache to start the proxy no matter what, but we schedule a new download of the updated lists later, as a background process, when the proxy will be started and the DNS service will work.

[iWARR]

Another ERROR:

I've restarted the service and got this one:

[CRITICAL] Unable use source [proxy v1 list from github]: [Invalid signature]
[NOTICE] Service restarted

[iWARR]

Thanks for this deep explanation. I like it.

The idea is that there is no source of truth any more.

Thats why I'm using DNSCrypt.

Instead of having a single list, you can subscribe to any number of lists. Each list can be hosted at a different location, and be signed with their own key.

If we have choice, this is nice. Same as tech options. I trust to you and your product. Ok. I will use your list, this is completely enough :) (And, of course, I've carefully choose the servers I connect to).

We use the (slightly outdated, but probably still okay) cache to start the proxy

Also, it was ability to use 2 services at once in the older DNSCrypt: Primary and Secondary. Now, DNSCrypt switches to alternative server "in series" (switches to the next, if expecting troubles with current server), isn't it?

A reasonable assertion is that the first time the proxy is started, the DNS settings haven't been changed to 127.0.0.1 yet. So, we can download the lists.

This is too idealistic situation. And have nothing common with real life :)

[jedisct1]

This has been partially implemented, as well as a few safeguards against servers possibly returning corrupted data.

What do you mean by "DNSCrypt"? If you are talking about dnscrypt-proxy, there was never any sort of "primary" or "secondary" as version 1 could only connect to a single server.

Version 2 can connect to as many servers as you like, and it will try to favor the fastest ones.

[jedisct1]

I released beta3 with a partial fix for the chicken-and-egg issue.

This is not a correct fix; the delay at startup time can be removed, and we need to schedule regular background updates. That will be for the next beta (which is unlikely to be before next week. Got quite a lot of actual work to catch up with).

[iWARR]

What do you mean by "DNSCrypt"? If you are talking about dnscrypt-proxy...

Yes, it is.

... there was never any sort of "primary" or "secondary" as version 1 could only connect to a single server.

Hm-m... In Windows I've insalled 2 DNS-services like this:

dnscrypt-proxy --install-with-config-file=DNSCrypt-Proxy-Primary.conf --service-name=DNSCrypt-Proxy-Primary

dnscrypt-proxy --install-with-config-file=DNSCrypt-Proxy-Secondary.conf --service-name=DNSCrypt-Proxy-Secondary

... and I had 2 *.conf files:

DNSCrypt-Proxy-Primary.conf DNSCrypt-Proxy-Secondary.conf

Long time ago I've started with "Simple DNSCrypt", that has 2 services, as well. Later I've moved to original software - dnscrypt-proxy. So, I decide, why I can't install 2 services in the dnscrypt-proxy also? That's funny story. And I've worked like this until present days :)

That's why I hope you should make a good documentation, at last. (I remember your roadmap ;)

And yes, from this old time I calling your "dnscrypt-proxy" as "DNSCrypt". I don't know why... Sorry for that :) Really, funny... Of course, I'd knew, that "Simple DNSCrypt" was based on your technology. GUI looks nice, however they can't fix main "core" bugs, have their own issues, and can't make progress as qiuckly as your development. Sadly. Anyway, I think "Simple DNSCrypt" may have a light future for the wide auditory, if developers will find more time and power for this nice software.

next beta (which is unlikely to be before next week. Got quite a lot of actual work to catch up with).

Great!

[iWARR]

Updated to Beta 3. Unfortunatelly, I still have the same [CRITICAL] message, like in the first post of this thread.

[elmakong]

guessing as the source file url path is on a domain name not IP address. i guess, by using the default server [servers.'dnscrypt.org-fr'] first then get the csv resource would resolve the problem?

update:

tried to update to beta 3 to check if resolved,

D:\Program Files\dnscrypt-proxy-win64-2.0.0beta3\win64>dnscrypt-proxy.exe -service install [2018-01-19 13:29:50] [NOTICE] Starting dnscrypt-proxy 2.0.0beta3 [2018-01-19 13:29:50] [CRITICAL] Unable use source [proxy v1 list from github]: [Get https://raw.githubusercontent.com/DNSCrypt/dnscrypt-resolvers/master/v1/dnscrypt-resolvers.csv: dial tcp: lookup raw.githubusercontent.com: no such host] [2018-01-19 13:29:50] [NOTICE] Installed as a service. Use -service start to start

D:\Program Files\dnscrypt-proxy-win64-2.0.0beta3\win64>dnscrypt-proxy.exe -service start [2018-01-19 13:30:02] [NOTICE] Starting dnscrypt-proxy 2.0.0beta3 [2018-01-19 13:30:02] [CRITICAL] Unable use source [proxy v1 list from github]: [Get https://raw.githubusercontent.com/DNSCrypt/dnscrypt-resolvers/master/v1/dnscrypt-resolvers.csv: dial tcp: lookup raw.githubusercontent.com: no such host] [2018-01-19 13:30:02] [NOTICE] Service started

but when i check with hostip (from dnscrypt v1):

D:\Program Files\dnscrypt-proxy-win64>hostip.exe -r 127.0.0.1 raw.githubusercontent.com 151.101.0.133 151.101.64.133 151.101.128.133 151.101.192.133

it replied the correct IPs

update 2: after running the service as above, the resource is loaded properly.

D:\Program Files\dnscrypt-proxy-win64-2.0.0beta3\win64>dnscrypt-proxy.exe -service restart [2018-01-19 13:46:48] [NOTICE] Starting dnscrypt-proxy 2.0.0beta3 [2018-01-19 13:46:49] [NOTICE] Source [https://raw.githubusercontent.com/DNSCrypt/dnscrypt-resolvers/master/v1/dnscrypt-resolvers.csv] loaded [2018-01-19 13:46:49] [NOTICE] Service restarted

[D1n0Bot]

I think it is ok because dnscrypt-proxy loaded the cached csv and service started. That’s why you are able to resolve the raw.githubusercontent.com and when you do a restart service, it is able to resolve the domain.

The solution is to pre resolve the domain in hosts file. Use the 4 IPs to resolve raw,githubusercontent.com.

You need to restart the pc.

[GrizzlyJr]

Temp solution, for the first time installed and started dnscrypt-proxy. Try to re-run dnscrypt-proxy.exe manually. it will download csv file first then exit, because port 53 already used.

[elmakong]

it may help or probably not, but here's my toml file

##############################################
#                                            #
#        dnscrypt-proxy configuration        #
#                                            #
##############################################

##################################
#         Global settings        #
##################################

## List of servers to use
## If this line is commented, all registered servers will be used

# server_names = ['cisco', 'fvz-anytwo', 'fvz-anyone']

## List of local addresses and ports to listen to. Can be IPv4 and/or IPv6.

listen_addresses = ['127.0.0.1:53']

## Require servers defined by remote sources to satisfy specific properties

# Server must support DNS security extensions
require_dnssec = false

# Server must not log user queries
require_nolog = true

# Server must not enforce its own blacklist (for parental control, ads blocking...)
require_nofilter = true

## Whether to the server as a background process (linux only)
## Do not set to true if you are using systemd

daemonize = false

## Always use TCP to connect to upstream servers

force_tcp = false

## Timeout, in milliseconds

timeout = 2500

## Delay, in minutes, after which certificates are reloaded

cert_refresh_delay = 300

#########################
#        Filters        #
#########################

## Immediately respond to IPv6-related queries with an empty response
## This makes things faster when there is no IPv6 connectivity

block_ipv6 = true

##################################################################################
#        Route queries for specific domains to a dedicated set of servers        #
##################################################################################

## Example map entries (one entry per line):
## example.com 9.9.9.9
## example.net 9.9.9.9,8.8.8.8

# forwarding_rules = 'forwarding-rules.txt'

###########################
#        DNS cache        #
###########################

## Enable a basic DNS cache to reduce outgoing traffic

cache = true

## Cache size

cache_size = 256

## Minimum TTL for cached entries

cache_min_ttl = 600

## Maxmimum TTL for cached entries

cache_max_ttl = 86400

## TTL for negatively cached entries

cache_neg_ttl = 60

###############################
#        Query logging        #
###############################

## Log client queries to a file

[query_log]

## Path to the query log file (absolute, or relative to the same directory as the executable file)

# file = 'query.log'

## Query log format (currently supported: tsv and ltsv)

format = 'tsv'

######################################################
#        Pattern-based blocking (blacklists)        #
######################################################

## Blacklists are made of one pattern per line. Example of valid patterns:
##
##   example.com
##   *sex*
##   ads.*
##   ads*.example.*
##   ads*.example[0-9]*.com

[blacklist]

## Path to the file of blocking rules (absolute, or relative to the same directory as the executable file)

# blacklist_file = 'blacklist.txt'

## Optional path to a file logging blocked queries

# log_file = 'blocked.log'

## Optional log format: tsv or ltsv (default: tsv)

# log_format = 'tsv'

#########################
#        Servers        #
#########################

## Remote lists of available servers
## Recommended: change the cache_file location to an absolute path

[sources]
  [sources.'proxy v1 list from github']
  url = 'https://raw.githubusercontent.com/DNSCrypt/dnscrypt-resolvers/master/v1/dnscrypt-resolvers.csv'
  minisign_key = 'RWQf6LRCGA9i53mlYecO4IzT51TGPpvWucNSCh1CBM0QTaLn73Y7GFO3'
  cache_file = 'dnscrypt-resolvers.csv'
  format = 'v1'
  refresh_delay = 168

## Local, static list of available servers

[servers]
  [servers.'cisco']
  provider_name = '2.dnscrypt-cert.opendns.com'
  address = '208.67.220.220'
  public_key = 'B735:1140:206F:225D:3E2B:D822:D7FD:691E:A1C3:3CC8:D666:8D0C:BE04:BFAB:CA43:FB79'

  [servers."fvz-anyone"]
  provider_name = '2.dnscrypt-cert.dnsrec.meo.ws'
  address = '185.121.177.177'
  public_key = '1A6A:D0A3:2B4C:5A61:A695:D153:670D:69AB:1690:3F9E:C3F7:F64F:13E5:35A3:18B2:28A5'

  [servers."fvz-anytwo"]
  provider_name = '2.dnscrypt-cert.dnsrec.meo.ws'
  address = '169.239.202.202'
  public_key = '1A6A:D0A3:2B4C:5A61:A695:D153:670D:69AB:1690:3F9E:C3F7:F64F:13E5:35A3:18B2:28A5'

[Tallefer]

I am glad that this bootstrap issue is finally brought up! I was thinking about opening a request to make a strictly non-DNS (https://xxx.xxx.xxx.xxx) solution (Is it possible? I think so) to bootstrap — download serverlist directly, hopefully minimizing exposure to MitM/DPI. I also thought that providing a TOR/I2P mirror would also help. But since the download.dnscrypt.org is no more, things are now more complicated...

[D1n0Bot]

Ah... it loads with your static server list. Even if first download of csv not successful, the dnscrypt-proxy still able to load up.. nice alternative.

[elmakong]

i guess the core function should load the static servers 1st then the remote sources..

[iWARR]

@Tallefer I'm agree with your arguments. We were talking with jedisct1 about this idea. But... I've tried https://xxx.xxx.xxx.xxx/... method, it doesn't work with GitHubUserContent, I suppose. I think, GitHub have some kind of complicated protection system to avoid such kind of access...

[Tallefer]

@iWARR > I've tried https://xxx.xxx.xxx.xxx/... method, it doesn't work with GitHubUserContent, Exactly. That was the first thing I've tried, along with download.dnscrypt.org . :) Thing is, from my (far from perfect) understanding of how Internet works — since the pool of pure IPv4 is dried out long ago, now servers commonly share one IP for several virtual hosts and, therefore, they cannot be accessed merely by their IP, without providing some additional info about destination. Which is carried along by DNS system, I guess... I'd be happy to see more correct explanation from anybody, though, if mine is wrong. :)

[Marco-vW]

I struggled with this same issue, but found a solution. I'm not sure how this could be solved on Windows, but in Linux (or actually my Asus router, running Asuswrt-Merlin) I have specified two servers (see below) for dnsmasq, which provides dns resolving and forwarding for my router by default. The following lines are added to it's config on boot.

no-resolv server=/pool.ntp.org/208.67.220.220 server=/raw.githubusercontent.com/208.67.220.220

The ntp-pool address is mandatory for when the firewall launches, as well as syncing correct time for validating certificates. The last line makes sure that dnscrypt-proxy is able to download its sources when it starts. This way dns resolving is available prior to dnscrypt-proxy starting and taken over by dnscrypt-proxy when it binds to a port.

Maybe editing C:\Windows\System32\Drivers\etc\hosts to predefine resolving raw.githubusercontent.com would suffice to solve your issue?

[jedisct1]

beta4 is out and should vastly improve this. It also improves the startup time in all situations. Give it a spin!

[iWARR]

@jedisct1 Not tested yet, but I see now double quotes in the *.toml again... # log_file = "dnscrypt-proxy.log"

instead: # log_file = 'dnscrypt-proxy.log'

Small note for the future update...

[sergeevabc]

Network connection is set to use 127.0.0.1 as primary DNS. The following snippet is added to dnscrypt-proxy.toml to solve the chicken-and-egg problem.

fallback_resolver = "77.88.8.88:1253" # Non-standard port to circumvent ISPs restriction
ignore_system_dns = true