IERS data format change

[ThatcherC]

I haven't been able to ingest IERS EOP parameters today:

julia> get_iers_eop_iau_1980()
ERROR: MethodError: Cannot `convert` an object of type SubString{String} to an object of type Float64

Closest candidates are:
  convert(::Type{T}, ::T) where T<:Number
   @ Base number.jl:6
  convert(::Type{T}, ::Ratios.SimpleRatio{S}) where {T<:AbstractFloat, S}
   @ Ratios ~/.julia/packages/Ratios/FsiCW/src/Ratios.jl:51
  convert(::Type{T}, ::Number) where T<:Number
   @ Base number.jl:7
  ...

Stacktrace:
...

This is with SatelliteToolbox v0.10.0 - I've been using that because the new versions break some functions I use in my existing programs.

I can confirm that the issue affects SatelliteToolbx v0.12.0 too though -

(@v1.10) pkg> st SatelliteToolbox
Status `~/.julia/environments/v1.10/Project.toml`
  [6ac157d9] SatelliteToolbox v0.12.0

julia> using SatelliteToolbo

julia> fetch_iers_eop(Val(:IAU1980))
[ Info: Downloading the file 'finals.all.csv' from 'https://datacenter.iers.org/data/csv/finals.all.csv'...
ERROR: MethodError: Cannot `convert` an object of type SubString{String} to an object of type Float64

Closest candidates are:
  convert(::Type{T}, ::T) where T<:Number
   @ Base number.jl:6
  convert(::Type{T}, ::T) where T
   @ Base Base.jl:84
  convert(::Type{T}, ::Number) where T<:Number
   @ Base number.jl:7
  ...

Stacktrace:

Turns out there was an announcement about this that I missed! https://datacenter.iers.org/data/2/message_490.txt

The start of the message explains why we are seeing this issue - new columns have been added to the CSV:

************************************************************************
IERS Message No. 490                                    October 09, 2023
************************************************************************

Changes to IERS EOP xml and csv formats and addition of JSON format

With the implementation of the IERS 2020 C04 system, the parameters
x rate, y rate, sigma x rate, and sigma y rate were also added to this
data series (see IERS Message No. 471).

In the IERS Data Center
(https://datacenter.iers.org/, https://data.iers.org/,
https://www.iers.org/IERS/EN/DataProducts/data.html),
these parameters will also be included in the xml, csv, and HTML files
starting November 6, 2023. This will result in a change to the xml and
csv formats for all EOP products on that date:

[...]

If you process csv files of any EOP products or the xml file of the
product "EOP 20 C04 (IAU2000A) - one file (1962-now)", you should check
if your routines still work correctly with the new format and adapt them
if necessary. For this purpose, we have provided sample files in the new
format at the following address:
https://datacenter.iers.org/products/eop/new_format_examples/

Please note: These files are only examples of the new format and will not
be updated.

Looks like the IERS fetching code needs to fixed! I'll probably put together a draft this afternoon and send it out. Is there any chance we can backport the change to older versions? I'm using v0.9.4 in and it'd be great if I could get the fix into that program without updating that program to the new SatelliteToolbox API.

[ronisbr]

Hi @ThatcherC !

Thanks for the report!

Sure! I think it will be very important to backport to v0.9.

[ThatcherC]

Awesome! I've got a super quick PR here - https://github.com/JuliaSpace/SatelliteToolboxTransformations.jl/pull/3

[ronisbr]

I will merge your PR as soon the tests finish. After that, I will back port the fix and tag a new v0.9.

[ThatcherC]

Awesome! Thanks for reviewing so quickly! And any chance it could also be backported to v0.10? I realized I have a few programs that pinned that version as well.

[ronisbr]

No problem! Let's do this :)

[ronisbr]

Hi @ThatcherC !

Sorry for the delay!

I tagged v0.1.5 of SatelliteToolboxTransformations.jl, and v0.9.5 and v0.10.1 of SatelliteToolbox.jl, which should add the support for the new format in all the ecosystem :)

Please, let me know if everything is working.

Thank you again for the report and the fix!

[ThatcherC]

Awesome! Thank you @ronisbr! I've updated all my programs and they're working great with v0.9.5 and v0.10.1! Thanks for the speedy backport! Here's hoping they don't change the IERS format again any time soon!