Support harmonic tide files

[kylecorry31]

• [ ] Add button to harmonic tide entry screen to choose from file (overwrites existing harmonics)
• [ ] CSV (custom)
• [ ] NOAA JSON (CO-OPs API)
• [ ] NOAA XML (CO-OPs API)

[Lee-Carre]

I'm going to address the slightly broader question of data-format for importing constituents for harmonic prediction.

I can see the appeal of CSV's simplicity & compactness. However, I feel that it may be significantly limiting and otherwise undesirable, for several reasons.

Firstly, I think it would be wise to determine if an existing format is already in use. There might be, from what I learned during my hunt for data sources (#1099).

Unless there's some major reason why CSV is clearly (long-term) advantageous.

Even if there isn't a formal standard; Kyle mentioned (#1068) that NOAA hosts an API which can serve responses that include HC data. Would it not be simpler to treat the output of said API as a de facto standard (I know of no other APIs for HC values) and mimic it for storing the same data? This avoids having to do format-conversion for every station. The process is then largely one of glorified copy-and-paste.

Beyond mere convenience, there may be reasons why an extensible format is safer.

In #1068 it was said that

In addition, the user will need to specify the data units (feet/meters) and Z0 offset (optional, defaults to MSL 0)

During my hunt, some sources (e.g. WOCE) supplied this metadata along with the set of constituents (all in the same machine-readable format & file). So, it would make sense to include that from the source.

Also published (as separate files) were confidence metrics of the HC values. Given the goal of #1103, including some kind of indication of the accuracy of the harmonic computed from the HCs would help this, instead of just assuming that HCs are always sound.

There was also what seemed to statistical / analytical data, about how the HCs were determined. I can't claim to understand it, though. I'm not sure if that would be helpful in selection of tidal location or not.

Another possibility, besides mimicking API responses, is to use (at least as a base) the format used by the largest source of HCs. WOCE hosted .txt files which seemed very parser-friendly to me.

There are temporal considerations, too.

• HCs may be refined in future, when more measurement data is available, or with better harmonic analysis
• other HC sets may be producing for the same tidal data; think of data scientists replicating a study of their peers
• MSL, relative to land, is likely to change; not only the sea itself (measured e.g. as radius from Earth's centre, or by GNSS attitude), but land rising / lowering due to geological factors. This was something made clear in the report I found when seeking data for my own island. The HCs presented for post-SL-rise, were different to the set for today
• as HCs are refined (either because the tidal pattern has changed, or from better measurement dataset & harmonic analysis), there may end up being multiple sets, each for a different time-range. In order to yield the ‘old’ predictions (for comparing against other sources) the historical sets of HCs need to be used. Even if one uses separate files for each set of HCs, there'll still need to be a way of specifying the date-range to which a given set apply

As for the possibility of an existing standardised format; amid reading (especially re WOCE) mention was made of the significant amount of data-exchange between participating institutions, for the project. Thus, it's likely that machine-readable formats were employed (or developed) to facilitate this. Not only for HCs, but also tidal measurement datasets, too (relevant for #1074).

I happened upon International Oceanographic Data Exchange, which I have not explored much. However, it would seem an obvious place to seek standardised formats, possibly the ones used by WOCE.

I would, thus, suggest that this issue be renamed (s/CSV harmonics/harmonics data-formats/) until a preferred format has been determined.

[kylecorry31]

Thanks, I will look at the format used by WOCE to see if that can be used here - if there is no standard though, I will use CSV - I wouldn't want to use the JSON/XML response from NOAA and would prefer a more human-readable format so the average person would be able to inspect the file downloaded from the website so they can confirm it is safe.

[aw-bib]

if there is no standard though, I will use CSV

It might be worthwhile to check the (binary) tcd format of xtide or equally well, their textual serialization. (The latter consists of two files customarily named HARMONICS and HARMONICS.IDX)

Simply, as one would have quite a few ports done already, so one could avoid the typing.