Cannot load.timescale()

JarleLand commented 5 years ago

Python 3.6.6 (default, Sep 12 2018, 18:26:19) [GCC 8.0.1 20180414 (experimental) [trunk revision 259383]] on linux Type "help", "copyright", "credits" or "license" for more information.

from skyfield import api from skyfield.api import load ts =api.load.timescale() Traceback (most recent call last): File "/usr/lib/python3.6/urllib/request.py", line 1318, in do_open encode_chunked=req.has_header('Transfer-encoding')) File "/usr/lib/python3.6/http/client.py", line 1239, in request self._send_request(method, url, body, headers, encode_chunked) File "/usr/lib/python3.6/http/client.py", line 1285, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/usr/lib/python3.6/http/client.py", line 1234, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/usr/lib/python3.6/http/client.py", line 1026, in _send_output self.send(msg) File "/usr/lib/python3.6/http/client.py", line 964, in send self.connect() File "/usr/lib/python3.6/http/client.py", line 936, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.6/socket.py", line 704, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.6/socket.py", line 745, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/jarle/.local/lib/python3.6/site-packages/skyfield/iokit.py", line 450, in download connection = urlopen(url) File "/usr/lib/python3.6/urllib/request.py", line 223, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python3.6/urllib/request.py", line 526, in open response = self._open(req, data) File "/usr/lib/python3.6/urllib/request.py", line 544, in _open '_open', req) File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain result = func(*args) File "/usr/lib/python3.6/urllib/request.py", line 1346, in http_open return self.do_open(http.client.HTTPConnection, req) File "/usr/lib/python3.6/urllib/request.py", line 1320, in do_open raise URLError(err) urllib.error.URLError: <urlopen error [Errno -2] Name or service not known>

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "", line 1, in File "/home/jarle/.local/lib/python3.6/site-packages/skyfield/iokit.py", line 269, in timescale data = self('deltat.data') File "/home/jarle/.local/lib/python3.6/site-packages/skyfield/iokit.py", line 179, in call download(url, path, self.verbose) File "/home/jarle/.local/lib/python3.6/site-packages/skyfield/iokit.py", line 452, in download raise IOError('cannot get {0} because {1}'.format(url, e)) OSError: cannot get http://maia.usno.navy.mil/ser7/deltat.data because <urlopen error [Errno -2] Name or service not known>

brandon-rhodes commented 5 years ago

It looks like the web site is down for me too! I'll go see if I can find any information about whether the server's name is changing.

brandon-rhodes commented 5 years ago

@JarleLand Oh — and, to get yourself moving again, visit this web search:

https://www.google.com/search?q="deltat.data"

If you click on the down-pointing triangle to the right of the first link, you can ask Google to show you their cached version of the file. If you save that to the directory where you're trying out your Python script, Skyfield should find it and not try to re-download it.

brandon-rhodes commented 5 years ago

Now that the outage has passed that was preventing its download, I'm still thinking about what the best approach will be so that Skyfield users don't depend on this single (if crucial) source of data.

gyth commented 5 years ago

I just found an ftp of "BUREAU INTERNATIONAL DES POIDS ET MESURES" that looks interesting for this question.

ftp://62.161.69.5/pub/tai/ or ftp://ftp2.bipm.org/pub/tai/

I admit I didn't look any deeper.... maybe this could be an additional source of data. maybe...

ghost commented 5 years ago

I also got lazy, but it appears https://www.iers.org/IERS/EN/DataProducts/EarthOrientationData/eop.html is the canonical (and non-US-government) source. Even http://maia.usno.navy.mil/ser7/readme refers to this as "IERS" data.

jsquyres commented 5 years ago

FWIW, I'm running into this problem this morning -- it looks like the IP name maia.usno.navy.mil does not resolve:

$ dig @8.8.8.8 maia.usno.navy.mil

; <<>> DiG 9.11.4-P2-3~bpo9+1-Debian <<>> @8.8.8.8 maia.usno.navy.mil
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 57295
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;maia.usno.navy.mil.        IN  A

;; Query time: 12 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Thu Dec 20 06:44:43 EST 2018
;; MSG SIZE  rcvd: 47

I was able to use the "download the cached version from Google" trick for deltat.data and deltat.preds files to get me going.

brandon-rhodes commented 5 years ago

Thanks for the update! Maybe this weekend I'll look at the IERS site and see if I can find a data file that's equivalent (unless of course one of you beats me to it!).

ghost commented 5 years ago

Thanks for the update! Maybe this weekend I'll look at the IERS site and see if I can find a data file that's equivalent (unless of course one of you beats me to it!).

Cough, cough! See my reply earlier :)

brandon-rhodes commented 5 years ago

Yes! I specifically meant "click through all of the data products on the page @barrycarter pointed to to see if this time (I've tried before) I can find a file with the same data, even if formatted differently, as the Navy's site."

ghost commented 5 years ago

My apologies. I thought that, since IERS released the data, it must be somewhere on their site and fairly easy to find.

I've looked deeper and I couldn't find the data anywhere there. All the links I've found for a canonical source for delta-t lead to nasa.gov.

There are some .edu "mirrors" but I don't know how current they are or how often they update (I suppose I could find out by looking at those files but haven't done so) or how reliable they are.

So, apparently, my link is not helpful, sorry.

ghost commented 5 years ago

This is probably unhelpful, but I finally reached http://toshi.nofs.navy.mil/ which lists the following backup servers (all .mil/.gov though):

http://maia.usno.navy.mil/ [main] ftp://maia.usno.navy.mil/ [main, FTP]

http://toshi.nofs.navy.mil/ ftp://toshi.nofs.navy.mil/

ftp://cddis.gsfc.nasa.gov/pub/products/iers

The last one may be most useful because it's a .gov not a .mil site.

jsquyres commented 5 years ago

FWIW, the name maia.usno.navy.mil became resolvable again (i.e., I checked today and it resolves).

I don't know if this is an extraordinary event or whether this happens not-infrequently (according to this issue, it's happened about twice in the span of a month, but that may or may not be indicative of a real pattern).

ghost commented 5 years ago

It was down for me for a couple of hours yesterday, but that's not necessarily indicative. Thoughts:

199.211.133.23 is maia's current IP address and may be quasi-fixed. You could try this when maia is down to see if it's a DNS issue (which it appears to be) and not a server issue.
alexa.com has info on navy.mil, but not maia.usno.navy.mil. It also doesn't appear to include uptime information anyway.
Several sites (like https://www.uptrends.com/tools/uptime) let you monitor a website's uptime (doesn't have to be your own) and might be useful for a serious test. As a note, the site is currently down for Beijing, Hong Kong, and Milan, but up otherwise.
If I understand correctly, both deltat.data and deltat.preds change much less than once a day. In theory, Brandon, you could rsync this to your repo daily and people could just pull from the skyfield source itself.

brandon-rhodes commented 5 years ago

This weekend, I'll see if I can put together a plan for these files to come built-in with Skyfield, with an API that makes it very clear to users that their predictions will go out of date if they don't regularly download new ones.

saufrecht commented 5 years ago

I encountered this error on Jan 18, 2019 with Leap_Second.dat. It appeared to hang for a minute, and then failed with:

ts = load.timescale()
OSError: cannot get https://hpiers.obspm.fr/iers/bul/bulc/Leap_Second.dat because <urlopen error [Errno 110] Connection timed out>

I tested connectivity from a different network, with the same result. This is not just a moved file, as https://hpiers.obspm.fr/ is equally unresponsive.

skyfield==1.9 Python 3.6.7

saufrecht commented 5 years ago

Temporary (until 28 June 2019) workaround for hpiers.obspm.fr outage: wget https://raw.githubusercontent.com/skyfielders/python-skyfield/master/ci/Leap_Second.dat

tmamedzadeh commented 5 years ago

@brandon-rhodes Hi! Is there an update?

verata-veritatis commented 5 years ago

Still having issues here, though I no longer think it's server-side. I'm able to access http://maia.usno.navy.mil/ser7/deltat.data without a problem in my browser, yet I'm still met with a urllib error in python:

raise IOError('cannot get {0} because {1}'.format(url, e)) OSError: cannot get http://maia.usno.navy.mil/ser7/deltat.data because <urlopen error [Errno 8] nodename nor servname provided, or not known>

Using 3.6.4.

brandon-rhodes commented 5 years ago

I have just released https://pypi.org/project/skyfield/1.11/ that supports load.timescale(builtin=True) that does not try to download updated files — enjoy!

swarnajyoti commented 4 years ago

Encountered the same problem with load.timescale() even if I installed skyfield 1.11. Any leads?

OSError: cannot get http://maia.usno.navy.mil/ser7/deltat.data because Remote end closed connection without response

skyfielders / python-skyfield

Cannot load.timescale() #218