Dealing with "duplicate" hazards from API.

[coreypieper]

"Duplicate" is in quotes, as the product described here is not technically a duplicate, but the manner in which it's produced results in multiple products in effect for the same geographic area set to expire at the same time - and all are supplied as active in CAP/API.

Following is the bundle created by @greg-does-weather's script capturing the instance of three air quality alerts in effect for the same area - each issued at different times (seemingly to update the headline so that temporal descriptions make more sense, i.e. ""AIR QUALITY ALERT IN EFFECT FROM 11 AM THIS MORNING TO 11 PM EDT THIS EVENING"" vs. "AIR QUALITY ALERT IN EFFECT FROM 11 AM TO 11 PM EDT FRIDAY"), but with the same expiration time (only real diff are API fields tied to when the product was issued): bundle_1718372690578.zip

Here is the output from the active endpoint at the same time: active.txt

It's worth noting that the legacy point and click only shows the most-recently-issued version of the product, though that dev team is unsure if this was intentional logic or a nice side-effect of another piece of logic.

Barring the API/CAP only supplying the most-recent version of these types of duplicates, perhaps it's possible to use logic with this project to only display the most recent version of products of the same type that apply to identical geographical areas and have the same expiration time.

[colinmurphy01]

Another instance of this in Cleveland.

bundle

[greg-does-weather]

There are (at least) two different issues causing duplicate alerts:

1. For alerts that do not originate at NWS (like air quality alerts), it seems that any update creates a new alert. (Note in the captured data from @coreypieper's message above, the messageType is Alert for all three air quality alerts, though they are clearly all the same alert. For updates, the messageType should be Update.)

   The ideal solution is for these updates to be noted as updates whenever they're ingested so they are handled correctly downstream. But, if we know which alerts originate outside of NWS, it seems totally reasonable to me for us to put in some logic to look at alert type, ending time, and coverage area, and assume that if those three things are the same between two alerts, they are the same. And in that case, use whichever one was issued most recently.

2. Alerts that have multiple segments get parsed by caphandler as multiple alerts. The API just ingests them directly. This seems correct: each segment has different information. The trouble is that the alert has a single polygon, and each segment is for a particular point within the polygon. There could be a really fun design exercise here around how best to display these, but it's important to point that there are indeed multiple distinct pieces of information in this case.

   Here's a good example. This is a flood advisory with multiple segments. The flood advisory is the single alert, and it covers a polygon defined along the course of a river. The segments provide additional detail at specific points along the river (eg, "Cumberland River near Bordeaux" vs. "Cumberland River near Hermitage").

   Note that in this case, the lat/lon of the point that a segment refers to is available in the original product, but it does not make it through caphandler to the API.

[sullynole]

"duplicates" can show up in API for any alert that does not originate with VTEC when the issuing office updates that alert in some way before the initial expiration. CapHandler dev will have additional insight into the CAP alerts that are created without P-VTEC. I know it happens with marine small craft/gale alerts from the Alaska WFOs CWFs as well.

[colinmurphy01]

See if this comes up in data workshop, then schedule follow up with cap handler team week of July 15th or later