IERS test updates failing

[sbailey]

unit tests are failing on the IERS download, since http://maia.usno.navy.mil/ser7/finals2000A.all is offline. Check whether this has moved somewhere else or whether we just need to wait for it to come back. I get slightly different errors on my laptop vs. NERSC, but one version is:

======================================================================
ERROR: test_update_iers (desisurvey.test.test_utils.TestUtils)
Test updating the IERS table.  Requires a network connection.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sbailey/anaconda3/envs/desi/lib/python3.6/urllib/request.py", line 1318, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/Users/sbailey/anaconda3/envs/desi/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Users/sbailey/anaconda3/envs/desi/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Users/sbailey/anaconda3/envs/desi/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Users/sbailey/anaconda3/envs/desi/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/Users/sbailey/anaconda3/envs/desi/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/Users/sbailey/anaconda3/envs/desi/lib/python3.6/http/client.py", line 936, in connect
    (self.host,self.port), self.timeout, self.source_address)
  File "/Users/sbailey/anaconda3/envs/desi/lib/python3.6/socket.py", line 704, in create_connection
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
  File "/Users/sbailey/anaconda3/envs/desi/lib/python3.6/socket.py", line 745, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 8] nodename nor servname provided, or not known. requested URL: http://maia.usno.navy.mil/ser7/finals2000A.all

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/sbailey/desi/git/desisurvey/py/desisurvey/test/test_utils.py", line 55, in test_update_iers
    utils.update_iers(save_name)
  File "/Users/sbailey/desi/git/desisurvey/py/desisurvey/utils.py", line 165, in update_iers
    iers = astropy.utils.iers.IERS_A.open(astropy.utils.iers.IERS_A_URL)
  File "/Users/sbailey/anaconda3/envs/desi/lib/python3.6/site-packages/astropy/utils/iers/iers.py", line 159, in open
    kwargs.update(file=download_file(file, cache=cache))
  File "/Users/sbailey/anaconda3/envs/desi/lib/python3.6/site-packages/astropy/utils/iers/iers.py", line 78, in download_file
    return utils.data.download_file(*args, **kwargs)
  File "/Users/sbailey/anaconda3/envs/desi/lib/python3.6/site-packages/astropy/utils/data.py", line 1088, in download_file
    raise e
  File "/Users/sbailey/anaconda3/envs/desi/lib/python3.6/site-packages/astropy/utils/data.py", line 1021, in download_file
    with urllib.request.urlopen(remote_url, timeout=timeout) as remote:
  File "/Users/sbailey/anaconda3/envs/desi/lib/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/Users/sbailey/anaconda3/envs/desi/lib/python3.6/urllib/request.py", line 526, in open
    response = self._open(req, data)
  File "/Users/sbailey/anaconda3/envs/desi/lib/python3.6/urllib/request.py", line 544, in _open
    '_open', req)
  File "/Users/sbailey/anaconda3/envs/desi/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/Users/sbailey/anaconda3/envs/desi/lib/python3.6/urllib/request.py", line 1346, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "/Users/sbailey/anaconda3/envs/desi/lib/python3.6/urllib/request.py", line 1320, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 8] nodename nor servname provided, or not known. requested URL: http://maia.usno.navy.mil/ser7/finals2000A.all>

[weaverba137]

See also desihub/desisim#391. There is no reason not to have a fallback data file cached to avoid this.

[dkirkby]

We already use a cached data file by default (via freeze_iers) to avoid problems like this and enforce a consistent model of future polar motions for coordinate transformations.

The failing unit test is exercising the code to occasionally update the cached data file, so it is useful to know that it is failing but this should have no impact on code that uses freeze_iers.

[dkirkby]

There is already an astropy issue for this: astropy/astropy#8304.

[weaverba137]

Ah, fair enough. While we have people's attention though, let me note that the desisim issue I referenced is still open. In other words, desisurvey is using freeze_iers, but every DESI package using it?

[sbailey]

@weaverba137 other packages are not yet using freeze_iers, but it hasn't been problematic enough to become a priority compared to other development issues.

The navy.mil site is back and tests are working again so I'll close this ticket. If this becomes problematic again in the future, see astropy/astropy#8308 for how they used the mirror site http://toshi.nofs.navy.mil/ser7/finals2000A.all