NTP time update delayed due to dead servers

[choksi81]

An Android packet trace shows that the node sits and blocks for minutes trying to resolve IP addresses for time servers that don't exist anymore. From a recent probe on Blackbox (sorted to show the problematic servers first):

Fail: nist1-dc.WiTime.net didn't resolve in 15.0707170963 seconds.
Fail: nist1-ny.WiTime.net didn't resolve in 16.6427960396 seconds.
Fail: nist1-sj.WiTime.net didn't resolve in 19.9971210957 seconds.
Fail: nist1.columbiacountyga.gov didn't resolve in 0.0442769527435 seconds.
Resolved time-a.nist.gov to 129.6.15.28 in 1.17663788795 seconds.
Resolved time-b.nist.gov to 129.6.15.29 in 0.000662088394165 seconds.
Resolved time-a.timefreq.bldrdoc.gov to 132.163.4.101 in 0.000824213027954 seconds.
Resolved time-b.timefreq.bldrdoc.gov to 132.163.4.102 in 0.000593900680542 seconds.
Resolved time-c.timefreq.bldrdoc.gov to 132.163.4.103 in 0.000629901885986 seconds.
Resolved utcnist.colorado.edu to 128.138.140.44 in 0.000577926635742 seconds.
Resolved time.nist.gov to 216.171.120.36 in 0.000999927520752 seconds.
Resolved time-nw.nist.gov to 131.107.13.100 in 0.000746011734009 seconds.
Resolved nist1.symmetricom.com to 69.25.96.13 in 0.000686883926392 seconds.
Resolved nist1.aol-ca.symmetricom.com to 207.200.81.113 in 0.0217320919037 seconds.
Resolved nist1.aol-va.symmetricom.com to 64.236.96.53 in 0.000623941421509 seconds.
Resolved nist.expertsmi.com to 50.245.103.198 in 0.000310182571411 seconds.
Resolved nist.netservicesgroup.com to 64.113.32.5 in 0.00183606147766 seconds.

Results on my machine are similar, fails are more in the 30 seconds range here.

Code:

import socket
import time

timeservers = ["time-b.nist.gov", "time-a.timefreq.bldrdoc.gov", "time-b.timefreq.bldrdoc.gov", "time-c.timefreq.bldrdoc.gov", "utcnist.colorado.edu", "time.nist.gov", "time-nw.nist.gov", "nist1.symmetricom.com", "nist1-dc.WiTime.net", "nist1-ny.WiTime.net", "nist1-sj.WiTime.net", "nist1.aol-ca.symmetricom.com", "nist1.aol-va.symmetricom.com", "nist1.columbiacountyga.gov", "nist.expertsmi.com", "nist.netservicesgroup.com"]("time-a.nist.gov",)

start = time.time()
for server in timeservers:
  try:
    address = socket.gethostbyname(server)
    print "Resolved", server, "to", address, "in", time.time()-start, "seconds."
    time.sleep(1)
  except:
    print "Fail:", server, "didn't resolve in", time.time()-start, "seconds."
  finally:
        start = time.time()

---

Quick fix: I'll remove the problematic servers from the server list. Long term fix: Figure out why gethostbyname blocks so incredibly long!