Add support for Harmonic Constituents for tides

[kylecorry31]

• [ ] Allow entry of either a simple or harmonic tide
• [ ] Indicate accuracy of the tide (simple vs harmonic) - this can be based on the number/type of constituents entered
• [x] Load and calculate harmonic tides

[kylecorry31]

The most important constituents appear to be:

• M2
• S2
• N2
• K1
• O1

But it should support the user adding more for better accuracy

In the US, constituents can be gathered here: https://tidesandcurrents.noaa.gov/harcon.html?unit=1&timezone=0&id=8455083&name=POINT+JUDITH%2C+HARBOR+OF+REFUGE&state=RI

Using GMT timezone

[kylecorry31]

In addition, the user will need to specify the data units (feet/meters) and Z0 offset (optional, defaults to MSL 0)

[Lee-Carre]

In the US, constituents can be gathered [from NOAA]

I tried to find sources of tidal data for Britain, which yielded the following:

• National Oceanography Centre
• National Tidal and Sea Level Facility

Annoyingly, it seems that not only aren't the harmonic constituents published, but even raw tide height measurements aren't available. Only the resulting predictions (for maybe 2 dozen days). (Mutters something unflattering & contemptuous about British attitudes toward data-hoarding, job-security, & control-freakery. Then something insulting about Ordinance Survey.) Of course, they're only too happy to sell licenses & consulting services, instead.

Seems that one is expected to both collect your own data, and then perform the harmonic analysis yourself.

What a joke, compared to NOAA. Sigh.

However, in my searching, I came across CORE (seemingly a British equivalent to ArXiv, hosting research papers), which hints at revealing more details in some of the documents.

[kylecorry31]

Thanks for the information!

Unfortunately due to the difficulty in obtaining harmonics, I will probably leave the Tides tool in the experimental mode. The simple tide clock uses a single high tide reference and is accurate to within 1 h 30 m during part of the lunar cycle. For most hiking purposes, that is fine, but it's just something that's hard to convey to users (real tide clocks have the same issue)

[Lee-Carre]

… due to the difficulty in obtaining harmonics, I will probably leave the Tides tool in the experimental mode.

Understandable.

It's admirable that your first wish is to implement a rigorous system that'll work well, long term (and entirely offline). Proper job, not half-arsed. Especially that one only needs to acquire the harmonic constituents once.

A suggestion, in that case: indicate, somewhere & somehow (maybe by pointing to a relevant GH issue) that tidal prediction remains experimental not because it's buggy or liable to crash the app, but because of lack of data needed to make it work. Besides informing users that it's safe to tinker with (outside of an emulator), it might encourage some to petition their nation's bureaucrats into publishing said data. Otherwise, they may never know.

Makes me wonder what arguments could be made, to acquire the data. Safety comes to mind, along with that no-one is profiting from TS which is available to everyone.

The simple tide clock uses a single high tide reference and is accurate to within 1 h 30 m …. For most hiking purposes, that is fine …

I was thinking similarly as a it'll-do-for-now hack.

However, since computed prediction tables are often available, how about allowing input of multiple (high and/or low) references, and using a somewhat less trivial algorithm.

While, of course, never as good as computing the astro harmonic, it might be a close-enough approximation (since, as you say, most expeditions are gonna be a few weeks, at the longest) until harmonic computation can be done more often. Increased accuracy, while still relatively simple. Should also work mostly anywhere.

[kylecorry31]

Thanks! When I originally implemented the tide clock (about a year ago), I went with the algorithm used by the traditional tide clocks of just setting a single high tide reference and using the average lunar cycle. I had tried to use the actual lunar day length but ended up with worse predictions than just the plain average. It is worth trying again since I know a lot more about tides and astronomy than a year ago.

I'll also look into inputting multiple high/low tide references.

[Lee-Carre]

… originally…, I went with the algorithm used by the traditional tide clocks of just setting a single high tide reference and using the average lunar cycle. I had tried to use the actual lunar day length but ended up with worse predictions …. It is worth trying again since I know a lot more ….

I'll also look into inputting multiple high/low tide references.

I've been tinkering with the tidal prediction feature, recently.

I was pleasantly surprised to find that it's actually quite accurate, given

• the simplicity of the implementation
• the documentation's warnings of ±90 minute uncertainties.

The prediction for half-a-lunar-cycle-later was only off by ~5 minutes 🙂👍.

Maybe that's

• due to where I am
• because of following the best-practice recommendation (match high-tide with lunar phase)
• luck / fortune (time of year?) 😄.

Aside: @kylecorry31; what's the tactic for most harshly stress-testing the simple predictor (while still giving it accurate data), to yield worst-case results? The documentation implies selecting a reference from when lunar phase is first quarter / last quarter. What about offset between reference & prediction? That is, the elapsed time between them. Though, I imagine further into the future will generally be worse (cumulative errors compounding). I'm wondering if there are particular portions or fractions of the lunar cycle which are more error-prone?

Further (strategic) thinking; since

• the true, long-term solution is computation of harmonics, especially for the tricky cases in which approximations & averages don't work well enough
• data sources for more-than-a-few locations have been discovered
• the simple tide clock is, at least, usable
• Kyle's development time is finite

then perhaps it is best to leave the existing implementation as-is in order to develop the harmonic-prediction implementation.

Basically, I see it as coming down to: the more time that's spent trying to refine the simple predictor, the longer it'll take to achieve harmonic-prediction. A desire for a refined simple predictor is, I think, ultimately due to not having harmonic prediction available.

Plus, when harmonic prediction is implemented, then the use-case problem becomes one of acquiring & using a set of harmonic constituents for the desired location. However, when users are then incentivised, their efforts in yielding one (be that via thorough searching, or performing harmonic analysis from a dataset) will benefit everyone.

Additionally, when harmonic computation is available, then a non-trivial fallback predictor becomes somewhat redundant.

I did think about withdrawing my suggestion / request for refining the simple predictor, however it would still be useful as a fallback (which works everywhere, requiring so little data that folks could measure it themselves) for those who are in places which don't yet have published harmonic constituents or aren't inclined / tech-savvy to do the (CSV) data-wrangling themselves.

[kylecorry31]

@Lee-Carre yes, I believe my testing used the date of first quarter to achieve the worst accuracy.

I also tried the following, and none of them achieved better results than the simple average lunar day:

1. Vary the time between tides based on the lunar day length
2. Predict the tide as an offset to lunar noon/directly below for the day (essentially the same as 1)
3. Factor in the solar noon offset (basically using computed M2 and S2)
4. Use the average duration between multiple high tides

I feel like the simple algorithm is as good as it can be - I imagine if there were a more accurate method out there, then the tide clocks would be using it (other than harmonics).

I also have #1039 planned for diurnal tides (mixed semi-diurnal are essentially semi-diurnal tides, with different heights of high/low tide during the day, so are covered already).

Harmonics are good for predicting the tides multiple days out, but the simple clock works well for most cases that TS supports (I'm still generating a use case document):

1. User wants to prepare for a hike a few days away - they could set the tide clock to a high tide on that day
2. User wants to know when the next low/high tide is when they know it is currently low/high (most likely for a few days maximum)

So maybe I can improve the documentation around how to get a more accurate prediction for a given day vs all time (all time being what it is currently recommending), and potentially researching a better algorithm to improve short term predictions (maybe the current lunar day duration) from the given day, and then falling back to the simple average lunar day. I'll have to do more research on that and get back to you.

[Lee-Carre]

I also tried the following, and none of them achieved better results than the simple average lunar day:

1. Vary the time between tides based on the lunar day length
2. Predict the tide as an offset to lunar noon/directly below for the day (essentially the same as 1)
3. Factor in the solar noon offset (basically using computed M2 and S2)
4. Use the average duration between multiple high tides

Most interesting. Thanks.

I happened upon the original issue for the tide clock (#466), and will read the article you cited there, to avoid guesswork of ignorance.

I feel like the simple algorithm is as good as it can be — I imagine if there were a more accurate method out there, then the tide clocks would be using it (other than harmonics).

Probably. Maybe not in physical clocks, but an algorithm would've been published somewhere (perhaps by mathematicians). While preference for accuracy would be nice, often manufacturers tend to go for what's cheapest / easiest. Like you say; to predict better (without kludges) is to compute harmonics from constituents. But that requires rather more than a mechanical clock.

On my own ignorance, & simple vs. harmonic prediction; reminds me of the convoluted corrections applied in attempt to make the geocentric model match observation. The solution wasn't to toil in refining it further, but to replace it with something fundamentally (pun not intended) better. In this case, that's harmonics.

So, yup, I'm more convinced that leaving the simple predictor akin to traditional tide clocks, is best. It remains familiar and useful for comparison, then. It's also reliable (in terms of users knowing what they're getting, known-limitations and all; unsurprising, robust).

Makes me extra glad that stubborn determination prevailed to find the data sources for driving harmonic prediction 😁. 5+ hours well-spent.

I also have #1039 planned for diurnal tides (mixed semi-diurnal are essentially semi-diurnal tides, with different heights of high/low tide during the day, so are covered already).

Ah. Diurnal tides was on my to-read-and-learn list.

Sounds like that would cover what I suggested previously, anyway. Good.

Harmonics are good for predicting the tides multiple days out

More than that, I gather.

In brief, the appeal of harmonics is akin to a perpetual calendar, or a watch that's powered by light and syncs time via radio signal (for the curious: Citizen NaviHawk). It just works, long-term. No need to keep correcting it. Problem perma-fixed.

So, now that there are enough data sources for harmonic prediction, this makes my original thinking re making-the-best-of-a-bad-situation redundant.

the simple clock works well for most cases that TS supports (I'm still generating a use case document):

1. User wants to prepare for a hike a few days away — they could set the tide clock to a high tide on that day
2. User wants to know when the next low/high tide is when they know it is currently low/high (most likely for a few days maximum)

So maybe I can improve the documentation around how to get a more accurate prediction for a given day vs all time (all time being what it is currently recommending), and potentially researching a better algorithm to improve short term predictions (maybe the current lunar day duration) from the given day, and then falling back to the simple average lunar day. I'll have to do more research on that and get back to you.

That sounds excellent. Much better bang-for-buck than my suggestion.

I do wonder if the algorithm for tide clocks assumes or relies on being set based on moon phase, precisely in order that the mechanics could remain simple (and thus more reliable, in a mechanical device). A physical tide clock likely comes with instructions that make this clear.

So, perhaps that algorithm was never intended to handle being set from a reference which isn't at new moon / full moon.

In which case (since a software version will have similar predictive characteristics), clear instructions (perhaps with links to material which explains why) are important.

Perhaps with a concluding note that much better predictions (harmonics) are in the pipeline.

Until harmonics becomes the standard in TS, it's as if there are 2 modes:

• easy, low-effort, moderate accuracy
• complicated, high-effort, very accurate

From a UX perspective, this then caters to different types of users (most being in the first group; the oddballs (myself included) are in the latter). Having both offers a path toward sophistication, for the sufficiently curious (which is often a mark of excellent software; rewarding effort without penalising those who are indifferent).

Eventually, a fully-implemented harmonic prediction system (that is, one which is highly usable and doesn't need extensive setup) will make things better for most users, anyway 🙂.

So, indeed; bring on the harmonics.

[kylecorry31]

Thanks! I would need to figure out why the simple tide clock algorithm recommends using the full/new moon high tide - almost all the instructions I found just said that would be the most accurate. I am planning on doing more research on the simple tide clock (probably during this development cycle), and am planning on implementing the harmonic tide clock for version 3.4.0 (a few months away)

[kylecorry31]

On its own, I don't really see allowing the user to enter harmonics as providing much value under TS' use cases, and here's why:

• If I'm able to enter the harmonic values into TS, then I have access to the Internet and can grab the tide tables that I want (which will likely be more accurate than TS). In this case, the preference is to use the tide tables rather than TS (possibly with the Notes tool in TS - maybe it would be beneficial to allow the user to enter a tide table, and show that as a graph, keeping the times they entered)
• If I'm on a trail (without Internet) and forgot a tide table, I wouldn't be able to enter harmonics so would need to make use of the simple tide clock (primary use case)
• Most users would not want to enter 20+ numbers/deal with downloading it from an external website for import into TS when they can more easily look up a pre-computed tide table

Adding support for harmonics will be useful though if #1102 is implemented (and would support the use cases: https://github.com/kylecorry31/Trail-Sense/wiki/Use-Cases#tides), I just don't see user entry of harmonics as being useful.

[Lee-Carre]

@kylecorry31

Interesting analysis.

There is a distinction between harmonic prediction as an implementation & tool, versus the UX & general usability in terms of being helpful to users when they need it (for decision-making, while on a skirmish).

As a technical tool, I see great advantage in it being sophisticated enough that once it has the HCs, it's then accurate for a very long time. Set-and-forget. The current predictor, even when seeded with a good reference, needs regular (monthly) updating kept it skew out of reasonable accuracy.

As for usability; indeed, inputting the HCs will not be a trivial task. However, that should remain a possibility (for the dedicated who really want it, even if it's by way of manually editing a structured-data file). Notably, it needs only be done once, to yield long-term benefits.

As you rightly say, with #1102 it's somewhat moot, since the problem is abstracted-away for most users. Only edge-cases will remain.

Hence, in other issues, my placing emphasis on there being a pipeline between computing HCs and those results ending up in the set to fulfil #1102. So that most users never have to even know about the underlying clockwork, and tidal prediction simply works for them.

Same as with discovering a clever / efficient algorithm, or answer to a difficult question; distribute that information so that others don't need to duplicate the effort of determining the same answer.

Bundling HCs means that the problem becomes one of occasionally adding new ones to TS (source), rather than something that the typical user must tackle.

[kylecorry31]

Okay - maybe to support those who really want it, I could add the upload of a structured file, but that will remain low priority for the reasons mentioned in my above comment. I don't think I will add an in-app UI to enter HCs.

I believe the work I'm doing on tide tables (#1124) better fits the use cases for TS, and thus will focus my development on that.