search

skyfielders / python-skyfield

Elegant astronomy for Python
MIT License
1.41k stars 211 forks source link

Downloads failing as maia.usno.navy.mil hs been shutdown #296

Closed jrs65 closed 4 years ago

jrs65 commented 4 years ago

As mentioned in IERS bulletin A (https://datacenter.iers.org/data/latestVersion/6_BULLETIN_A_V2013_016.txt) the USNO server that Skyfield uses to get deltat.data etc. has been shutdown as of today until April next year. Probably Skyfield should switch to one of the other servers like the recommended ftp://cddis.nasa.gov/products/iers/

Deuchnord commented 4 years ago

The deltat.preds on NASA’s FTP seems to be unparsable:

  File "/home/jerome/.local/share/virtualenvs/kosmorro--p-Z3VDj/lib/python3.7/site-packages/skyfield/iokit.py", line 302, in timescale
    preds = self('deltat.preds')
  File "/home/jerome/.local/share/virtualenvs/kosmorro--p-Z3VDj/lib/python3.7/site-packages/skyfield/iokit.py", line 196, in __call__
    expiration_date, data = parser(f)
  File "/home/jerome/.local/share/virtualenvs/kosmorro--p-Z3VDj/lib/python3.7/site-packages/skyfield/iokit.py", line 389, in parse_deltat_preds
    year_float, delta_t = np.loadtxt(lines, usecols=[1, 2]).T
  File "/home/jerome/.local/share/virtualenvs/kosmorro--p-Z3VDj/lib/python3.7/site-packages/numpy/lib/npyio.py", line 1146, in loadtxt
    for x in read_data(_loadtxt_chunksize):
  File "/home/jerome/.local/share/virtualenvs/kosmorro--p-Z3VDj/lib/python3.7/site-packages/numpy/lib/npyio.py", line 1074, in read_data
    items = [conv(val) for (conv, val) in zip(converters, vals)]
  File "/home/jerome/.local/share/virtualenvs/kosmorro--p-Z3VDj/lib/python3.7/site-packages/numpy/lib/npyio.py", line 1074, in <listcomp>
    items = [conv(val) for (conv, val) in zip(converters, vals)]
  File "/home/jerome/.local/share/virtualenvs/kosmorro--p-Z3VDj/lib/python3.7/site-packages/numpy/lib/npyio.py", line 781, in floatconv
    return float(x)
ValueError: could not convert string to float: 'TT-UT'

Loos like it’s trying to parse the file headers. Tried to remove it, but then Skyfield backs the file up and tries to download a new version.

jrs65 commented 4 years ago

@Deuchnord, I was finding the same issue the other day when I starting putting together a PR to fix this one. Didn't have time to investigate a fix unfortunately.

jrs65 commented 4 years ago

Okay. Not only are the headers different, but the columns are different, and the data seems somewhat out of date.

Here's the start of the old data from the USNO:

   MJD        YEAR    TT-UT Pred  UT1-UTC Pred  ERROR
   58484.000  2019.00   69.34      -0.152       0.117
   58575.000  2019.25   69.48      -0.295       0.162
   58666.000  2019.50   69.62      -0.440       0.215
   58758.000  2019.75   69.71      -0.527       0.273
   58849.000  2020.00   69.87                   0.335
   58940.000  2020.25   70.03                   0.399
   59032.000  2020.50   70.16                   0.465
   59123.000  2020.75   70.24                   0.532
   59214.000  2021.00   70.39                   0.600

and here is the new data from NASA CDDIS:


  YEAR    TT-UT PREDICTION  UT1-UTC PREDICTION  ERROR

 2015.75      67.97               0.210         0.02
 2016.00      68.11               0.079         0.06
 2016.25      68.2                              0.1
 2016.50      68.4                              0.2
 2016.75      68.5                              0.2
 2017.00      68.6                              0.3
jrs65 commented 4 years ago

Okay. Not even the delta.data file is up to date. The last entry in the file is from 2019/01. I'm going to try and give them an email about it all.

jrs65 commented 4 years ago

Alright. I'm having a bit of a back and forth with the CDDIS folks now. I'll post again if there's any resolution.

Deuchnord commented 4 years ago

Looks like the files have been updated yesterday evening :)

jrs65 commented 4 years ago

Yeah. I was just about to post about that. The CDDIS person I was talking with pinned it down to an issue at the USNO end, it sounds like something was wrong with what they were pushing. In any case it is indeed back, and the deltat.preds is back in the expected format, so it should just work.

Deuchnord commented 4 years ago

Yes :+1: I'm preparing a PR to fix the download of the files :)

jrs65 commented 4 years ago

I've actually just sent in a PR. So probably no need if you think the changes are okay.

brandon-rhodes commented 4 years ago

Thanks for working on this while I was traveling! I have just released a new version of Skyfield (1.14) that should pull from the new URL.