The default reflector list consists entirely of DNS servers

[patrakov]

As the subject says, the default reflector list consists entirely of DNS servers - probably because these are the obvious subjects of anycast setups. However, this project is not the only one that maintains such a list. Various oppressive governments around the world request ISPs to make certain sites inaccessible to the public, and ISPs who implement that using DNS-based filtering see that alternative DNS servers make a loophole in this plan. Even though we send no DNS queries to these servers, only pings, this still risks hitting the "all default reflector IPs are hijacked by the ISP" scenario, and I have seen this.

To mitigate this risk, please add a few reflectors that do not run a DNS server and are not seen as censorship circumvention devices - enough so that the project is still usable by default with such kind of censorship.

[rany2]

Suggestion: use Google, Cloudflare, Facebook, MSFT, Apple and pool.ntp.org's NTP time servers.

[moeller0]

No, please, let's not add NTP servers, these can be quite picky in what they consider abuse. The actual NTP protocol has methods by which a NTP server can tell a client to reduce its query frequency (and IIRC even to bugger off) exactly so NTP servers can manage their load. That said, if we would run our own NTP server pool, we could easily use those as reflectors (however in that case we should switch to using the NTP protocol to get high quality synchronized absolute time).

[rany2]

The actual NTP protocol has methods by which a NTP server can tell a client to reduce its query frequency (and IIRC even to bugger off) exactly so NTP servers can manage their load.

What I meant here is that we'd simply be doing ICMP pings on these NTP server addresses.

[moeller0]

Yes, I understand what you meant, but that is still not OK in my opinion. For a NTP server to operate well it needs to make sure not to be overloaded, so a bit of carefulness seems advised.

[rany2]

Fair enough, I'm just not able to think of another service which would satisfy this condition:

please add a few reflectors that do not run a DNS server and are not seen as censorship circumvention devices

I guess we could just pick servers that aren't low latency... Don't see why we need them to be anycast/low latency.

[rany2]

I'm not sure how great of an idea this is but what about running a traceroute to any IP and using the second hop as the reflector?

[moeller0]

Might work, might not, the early infrastructure hops of my ISP (unlike its DNS servers) are rather capricious and will drop the occasionally ICMP response and/or return some probes with excessive latencies. Sure cake-autorate is trying to control for all of that, but put frankly these are sub-optimal reflectors (that might well be different for other ISPs, but rate-limiting and deprioritization of ICMP are well described behavior for infrastructure nodes).

I think @dlakelan proposed something earlier similar to your idea about testing the first couple of hops for suitability.

I guess maybe we could see whether we can whip a simply test script that could be used by users in the same situation @patrakov described to get a set of candidate reflectors...

[lynxthecat]

I did start writing a script at one point to test for suitable reflectors by working through @tievolu's huge reflector list:

#!/bin/bash

initial_reflector_set_url=https://raw.githubusercontent.com/tievolu/timestamp-reflectors/main/reflectors-europe.csv

select_reflectors()
{
    for ((retries=0; retries<3; retries++))
    do
        wget -O /tmp/initial_reflector_set https://raw.githubusercontent.com/tievolu/timestamp-reflectors/main/reflectors-europe.csv
        [[ $? -eq 0 ]] && break
                sleep 5
    done

    declare -A reflector_rtt

    while read reflector
    do
        [[ $reflector =~ ([0-9]+.[0-9]+.[0-9]+.[0-9]) ]] || continue
        reflector=${BASH_REMATCH[1]}
        echo "testing $reflector"
        [[ $(ping -q -c 5 -i 0.1 "$reflector" | tail -1) =~ ([0-9.]+)/ ]] && printf -v reflector_rtt[$reflector] %.0f\\n "${BASH_REMATCH[1]}e3" || reflector_rtt[$reflector]=1000
        echo ${reflector_rtt[$reflector]}
    done</tmp/initial_reflector_set 
}

select_reflectors

But taking this forward would require more work, and I don't feel especially motivated to work on this since the default reflector list based on DNS servers seems to work well for most people.

Does anyone feel motivated to develop the script above or have any other suggestions? Or shall I just close this issue and provide some guidance in the README?

[rany2]

Or shall I just close this issue and provide some guidance in the README?

I suggest that if the script isn't able to get ANY response from all of the reflectors it prints some message explaining that the reflector list needs to be changed. Maybe then it would link to your README section.

Edit: of course, it shouldn't be a fatal error.

[rany2]

initial_reflector_set_url=https://raw.githubusercontent.com/tievolu/timestamp-reflectors/main/reflectors-europe.csv

I don't like that list you've linked to because it is just too large to be useful. We want quality reflectors not ALL servers on the internet that support ICMP timestamp requests.

[lynxthecat]

I don't like that list you've linked to because it is just too large to be useful. We want quality reflectors not ALL servers on the internet that support ICMP timestamp requests.

I posted the code snippet only to demonstrate a way to determine a possible set of viable reflectors. Of course it would be possible to just randomly pick a few hundred and probe those. Or something else.

I suggest that if the script isn't able to get ANY response from all of the reflectors it prints some message explaining that the reflector list needs to be changed. Maybe then it would link to your README section.

At the moment if no reflectors responds then warnings will get posted to the system log:

https://github.com/lynxthecat/cake-autorate/blob/680e9954ba6e5af923ca792d1ecfa53bd5feac6e/cake-autorate.sh#L2306

---

In general, it seems from the above discussion that there is no easy way to expand on the default reflector set in a way that addresses @patrakov's issue.

[rany2]

You might want to look into https://www.iana.org/domains/root/servers as these are also anycast. I doubt that those ever get banned.

[patrakov]

CDNs such as Cloudflare or Fastly use anycast and thus give low latencies. Maybe we could add a few of their nodes?

[lynxthecat]

Can you suggest a list @patrakov?

[patrakov]

Cloudflare IP ranges: https://www.cloudflare.com/ips/ - however, these are ranges, not IPs. Trying to ping random ones resulted in no response, however, the results from this DuckDuckGo dork worked: "cloudflare site:netify.ai" and resulted in a few reflectors, all of them with 40-50 ms latency from the Philippines. I don't know how stable they are. They are not suitable for use with tsping.

104.16.226.10
104.17.127.171
104.18.42.171
104.22.5.69
162.159.128.7
162.159.129.232
172.66.0.60
172.67.183.37
188.114.96.7
188.114.97.0

EDIT: I no longer have access to the network that prompted the initial request, so can't test if they work there.

A similar experiment with Fastly was not successful - they use DNS-based steering of the client to the nearest point of presence, not anycast, so the reflectors found by DuckDuckGo exhibit high latency.

Other presumably-anycast CDNs that can help are AWS Global Accelerator, Apple CDN, Azure Front Door.

[lynxthecat]

Well I see responses to those IPs, but I have no way of knowing if these should be added to the default list.

Perhaps I should just add a short section to the INSTALLATION page that instructs the user to verify that responses are seen to the list of reflectors present in the config, and that it is the responsibility of the user to change these to more appropriate alternatives if not. Is there a generic way to obtain random IPs? I don't know what a DuckDuckGo dork is.

[moeller0]

Maybe just add them to your set and see whether they get used or sorted out quickly? After all you taught autorate to retire bad reflectors so testing it should be easy? Maybe add another file to store which reflectors got retired and for which reason (random replacement is fine, being sorted out because failing one of the sanity tests would be 'strike' against a reflector?

[moeller0]

Randomising IPs is really easy, randomising decent candidates less so ;)

[lynxthecat]

I just added some further instructions in the README that might help with this and configuration in general.

[patrakov]

I think that the issue has been addressed to a sufficient degree. Thanks!

[Wabuo]

What's with game servers?

As in a bunch of the for example battel-net servers?

I'm sure they get pinged all the time for latency tests.

[moeller0]

If you know ip addresses or names of actual game servers that respond, and preferentially to both ICMP echo and ICMP timestamp requests, please post them here. From my limited experience trying to help debug bad gaming experiences, it looked like most actual game servers do not respond to such requests at all.