USNO files server look down

[brunobord]

How to reproduce

>>> from skyfield import api
>>> ts = api.load.timescale()
Traceback (most recent call last):
  File "/usr/lib/python3.6/urllib/request.py", line 1318, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/usr/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/usr/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/usr/lib/python3.6/http/client.py", line 936, in connect
    (self.host,self.port), self.timeout, self.source_address)
  File "/usr/lib/python3.6/socket.py", line 704, in create_connection
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
  File "/usr/lib/python3.6/socket.py", line 745, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/novapost/.virtualenvs/EXP3/lib/python3.6/site-packages/skyfield/iokit.py", line 477, in download
    connection = urlopen(url)
  File "/usr/lib/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.6/urllib/request.py", line 526, in open
    response = self._open(req, data)
  File "/usr/lib/python3.6/urllib/request.py", line 544, in _open
    '_open', req)
  File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 1346, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "/usr/lib/python3.6/urllib/request.py", line 1320, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno -2] Name or service not known>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/novapost/.virtualenvs/EXP3/lib/python3.6/site-packages/skyfield/iokit.py", line 275, in timescale
    data = self('deltat.data')
  File "/home/novapost/.virtualenvs/EXP3/lib/python3.6/site-packages/skyfield/iokit.py", line 185, in __call__
    download(url, path, self.verbose)
  File "/home/novapost/.virtualenvs/EXP3/lib/python3.6/site-packages/skyfield/iokit.py", line 479, in download
    raise IOError('cannot get {0} because {1}'.format(url, e))
OSError: cannot get http://maia.usno.navy.mil/ser7/deltat.data because <urlopen error [Errno -2] Name or service not known>

Notes

1. It looks like the whole service is down => https://downforeveryoneorjustme.com/maia.usno.navy.mil
2. there's no flag available to avoid trying to download up-to-date files, so the program crashes
3. the latest deltat.data file that we can find on this repository looks obsolete

Are there any mirrors available or alternative data sources?

[brunobord]

... and it's back up.

I feel silly, now, I've tried for about 30mn and now it works and downloads up-to-date files.

Before I hit "close" for this issue, would you confirm that I could avoid trying to download the files by using a custom Loader like this?

load = Loader('.', expire=False)
ts = load.timescale()

[brandon-rhodes]

I'm planning for the next version of Skyfield to switch to carrying those files inline, and by default to use the inline files unless the user insists that they need to be brought up to date. I'll comment here on this issue once a new version is out that includes that feature! Thanks for reporting the difficulty.

[brunobord]

excellent! thank you very much!

[HelloWorldIL]

It seems like https://maia.usno.navy.mil/ is blocked in many counties outside the US: https://asm.ca.com/en/checkit.php Here in Israel Skyfield is essentially unusable without a VPN. This issue has to be fixed asap.

[brandon-rhodes]

Could someone try installing Skyfield with:

pip install https://github.com/skyfielders/python-skyfield/archive/master.zip

and see if you can now load a timescale with load.timescale(builtin=True) and avoid needing to download USNO files? Thanks!

[HelloWorldIL]

Hi, thanks for the quick reply! I tried running load.timescale(builtin=True) on a fresh ubuntu docker container with no error, but when I tried loading the current time with load.timescale(buitin=True).now() I got the following error:

`>>> ts.now() Traceback (most recent call last): File "/usr/local/lib/python3.6/dist-packages/numpy/core/fromnumeric.py", line 56, in _wrapfunc return getattr(obj, method)(*args, **kwds) AttributeError: 'datetime.date' object has no attribute 'searchsorted'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python3.6/dist-packages/skyfield/timelib.py", line 89, in now return self.utc(self._utcnow().replace(tzinfo=utc)) File "/usr/local/lib/python3.6/dist-packages/skyfield/timelib.py", line 110, in utc tai = _utc_datetime_to_tai(self.leap_dates, self.leap_offsets, dt) File "/usr/local/lib/python3.6/dist-packages/skyfield/timelib.py", line 848, in _utc_datetime_to_tai hour, minute, second + dt.microsecond / 1000000.00) File "/usr/local/lib/python3.6/dist-packages/skyfield/timelib.py", line 856, in _utc_to_tai i = searchsorted(leap_dates, j, 'right') File "/usr/local/lib/python3.6/dist-packages/numpy/core/fromnumeric.py", line 1246, in searchsorted return _wrapfunc(a, 'searchsorted', v, side=side, sorter=sorter) File "/usr/local/lib/python3.6/dist-packages/numpy/core/fromnumeric.py", line 66, in _wrapfunc return _wrapit(obj, method, *args, *kwds) File "/usr/local/lib/python3.6/dist-packages/numpy/core/fromnumeric.py", line 46, in _wrapit result = getattr(asarray(obj), method)(args, **kwds) ValueError: object of too small depth for desired array`

I get the same error when executingload.timescale(buitin=True).utc(year=2001)

When running the command without buitin=True I get no errors.

Using Python 3

[brandon-rhodes]

@HelloWorldIL Thanks! I have turned your example code into a formal test and fixed the issue. Could you try again?

[HelloWorldIL]

@brandon-rhodes Repeating the test and executing load.timescale(builtin=True).now() on a fresh ubuntu container with Skyfield installed via the command pip3 install https://github.com/skyfielders/python-skyfield/archive/master.zip, I now get no errors! This is fantastic for anybody using Skyfield outside the US and for anytime the servers are down. Thank you!

[brandon-rhodes]

Wonderful! I'll do a release today to make the feature available.

[brandon-rhodes]

I have just released https://pypi.org/project/skyfield/1.11/ — enjoy!

[HelloWorldIL]

@brandon-rhodes Thanks!