search

nmlorg / metabot

Modularized, multi-account bot.
https://metabot.readthedocs.io/
5 stars 0 forks source link

Add temperature, driving conditions, storm warning info to event summaries #80

Open nmlorg opened 4 years ago

nmlorg commented 4 years ago
  1. Maintain a cache of location string to lat/lon. Whenever an unknown location is seen, use an API like Google Geocoding API to look it up.
  2. Maintain a cache of lat/lon to forecast grid point using https://api.weather.gov/points/LAT,LON. Note that lat and lon must be truncated to 4 decimal points.
  3. Whenever an event is displayed, look up its forecast grid point's forecast for the given time. This might just be done in m.m.events.format_event the same way ticketing information is checked. It might make sense to introduce some kind of caching layer here that only exists for the duration of the current request (something shoved into ctx?).

Note that weather.gov's API provides two forecast streams:  hourly and daily. For example, Capitol Hill, Seattle, WA geocodes (1) to:

{
   "results" : [
      {
         "geometry" : {
            "location" : {
               "lat" : 47.625305,
               "lng" : -122.3221835
            }
         }
      }
   ]
}

which maps to forecast grid point (2):

{
    "properties": {
        "forecast": "https://api.weather.gov/gridpoints/SEW/124,68/forecast",
        "forecastHourly": "https://api.weather.gov/gridpoints/SEW/124,68/forecast/hourly",
    }
}

which provides forecasts for2019-11-29T18:00:00-08:00 through 2019-12-06T18:00:00-08:00 in 12-hour blocks and 2019-11-29T18:00:00-08:00 through 2019-12-06T06:00:00-08:00 in 1-hour blocks, resp.; so if we had an event taking place at noon on Dec 6, it might make sense to go with the 12-hour forecast. (Though in practice it might not be worth the effort if the extra coverage is consistently so small.)

nmlorg commented 4 years ago

Note that only the daily (12-hourly) forecast includes free-form description text, and only the raw gridpoints endpoint seems to include weather advisories:

https://api.weather.gov/gridpoints/MPX/107,71/forecast/hourly

            {
                "number": 1,
                "name": "",
                "startTime": "2019-11-29T20:00:00-06:00",
                "endTime": "2019-11-29T21:00:00-06:00",
                "isDaytime": false,
                "temperature": 32,
                "temperatureUnit": "F",
                "temperatureTrend": null,
                "windSpeed": "10 mph",
                "windDirection": "ESE",
                "icon": "https://api.weather.gov/icons/land/night/snow_fzra,30?size=small",
                "shortForecast": "Chance Light Snow",
                "detailedForecast": ""
            },

 

https://api.weather.gov/gridpoints/MPX/107,71/forecast

            {
                "number": 1,
                "name": "Tonight",
                "startTime": "2019-11-29T20:00:00-06:00",
                "endTime": "2019-11-30T06:00:00-06:00",
                "isDaytime": false,
                "temperature": 31,
                "temperatureUnit": "F",
                "temperatureTrend": null,
                "windSpeed": "10 to 20 mph",
                "windDirection": "E",
                "icon": "https://api.weather.gov/icons/land/night/snow_fzra,100/sleet,100?size=medium",
                "shortForecast": "Light Snow",
                "detailedForecast": "A chance of snow and a slight chance of freezing rain before 9pm, then snow and a slight chance of thunderstorms between 9pm and 5am, then snow and a chance of sleet and a slight chance of thunderstorms. Cloudy, with a low around 31. East wind 10 to 20 mph, with gusts as high as 25 mph. Chance of precipitation is 100%. New snow accumulation of 1 to 3 inches possible."
            },

 

https://api.weather.gov/gridpoints/MPX/107,71

        "hazards": {
            "values": [
                {
                    "validTime": "2019-11-30T03:00:00+00:00/P1DT15H",
                    "value": [
                        {
                            "phenomenon": "WS",
                            "significance": "W",
                            "event_number": "2"
                        }
                    ]
                }
            ]
        },
nmlorg commented 4 years ago

Poking around some more, it looks like we might want to take the forecastZone field from https://api.weather.gov/points/LAT,LON, strip everything before the last '/', then use that to pull advisories directly from api.weather.gov/alerts. For example, Capitol Hill's full properties definition is:

    "properties": {
        "@id": "https://api.weather.gov/points/47.6253,-122.3221",
        "@type": "wx:Point",
        "cwa": "SEW",
        "forecastOffice": "https://api.weather.gov/offices/SEW",
        "gridX": 124,
        "gridY": 68,
        "forecast": "https://api.weather.gov/gridpoints/SEW/124,68/forecast",
        "forecastHourly": "https://api.weather.gov/gridpoints/SEW/124,68/forecast/hourly",
        "forecastGridData": "https://api.weather.gov/gridpoints/SEW/124,68",
        "observationStations": "https://api.weather.gov/gridpoints/SEW/124,68/stations",
        "relativeLocation": {
            "type": "Feature",
            "geometry": {
                "type": "Point",
                "coordinates": [
                    -122.350876,
                    47.620499000000002
                ]
            },
            "properties": {
                "city": "Seattle",
                "state": "WA",
                "distance": {
                    "value": 2221.7450616248334,
                    "unitCode": "unit:m"
                },
                "bearing": {
                    "value": 76,
                    "unitCode": "unit:degrees_true"
                }
            }
        },
        "forecastZone": "https://api.weather.gov/zones/forecast/WAZ558",
        "county": "https://api.weather.gov/zones/county/WAC033",
        "fireWeatherZone": "https://api.weather.gov/zones/fire/WAZ654",
        "timeZone": "America/Los_Angeles",
        "radarStation": "KATX"
    }

and https://api.weather.gov/alerts/active?zone=WAZ558 currently gives:

{
    "type": "FeatureCollection",
    "features": [],
    "title": "current watches, warnings, and advisories for Seattle and Vicinity (WAZ558) WA",
    "updated": "2019-11-24T00:00:00+00:00"
}

  Minneapolis, MN is currently under a winter storm warning, being pushed to Android devices linking back to https://www.google.org/publicalerts/alert?aid=f51afc5f2c0d1ae9. Its full properties definition is:

    "properties": {
        "@id": "https://api.weather.gov/points/44.9777,-93.265",
        "@type": "wx:Point",
        "cwa": "MPX",
        "forecastOffice": "https://api.weather.gov/offices/MPX",
        "gridX": 107,
        "gridY": 71,
        "forecast": "https://api.weather.gov/gridpoints/MPX/107,71/forecast",
        "forecastHourly": "https://api.weather.gov/gridpoints/MPX/107,71/forecast/hourly",
        "forecastGridData": "https://api.weather.gov/gridpoints/MPX/107,71",
        "observationStations": "https://api.weather.gov/gridpoints/MPX/107,71/stations",
        "relativeLocation": {
            "type": "Feature",
            "geometry": {
                "type": "Point",
                "coordinates": [
                    -93.268320000000003,
                    44.963324
                ]
            },
            "properties": {
                "city": "Minneapolis",
                "state": "MN",
                "distance": {
                    "value": 1619.735883443921,
                    "unitCode": "unit:m"
                },
                "bearing": {
                    "value": 9,
                    "unitCode": "unit:degrees_true"
                }
            }
        },
        "forecastZone": "https://api.weather.gov/zones/forecast/MNZ060",
        "county": "https://api.weather.gov/zones/county/MNC053",
        "fireWeatherZone": "https://api.weather.gov/zones/fire/MNZ060",
        "timeZone": "America/Chicago",
        "radarStation": "KMPX"
    }

and https://api.weather.gov/alerts/active?zone=MNZ060 currently gives:

{
    "@context": [
        "https://raw.githubusercontent.com/geojson/geojson-ld/master/contexts/geojson-base.jsonld",
        {
            "wx": "https://api.weather.gov/ontology#",
            "@vocab": "https://api.weather.gov/ontology#"
        }
    ],
    "type": "FeatureCollection",
    "features": [
        {
            "id": "https://api.weather.gov/alerts/NWS-IDP-PROD-3935103-3346792",
            "type": "Feature",
            "geometry": null,
            "properties": {
                "@id": "https://api.weather.gov/alerts/NWS-IDP-PROD-3935103-3346792",
                "@type": "wx:Alert",
                "id": "NWS-IDP-PROD-3935103-3346792",
                "areaDesc": "Carver; St. Croix; McLeod; Ramsey; Kandiyohi; Wright; Meeker; Dunn; Washington; Chippewa; Hennepin; Anoka",
                "geocode": {
                    "UGC": [
                        "MNZ068",
                        "WIZ023",
                        "MNZ066",
                        "MNZ062",
                        "MNZ057",
                        "MNZ059",
                        "MNZ058",
                        "WIZ025",
                        "MNZ063",
                        "WIZ027",
                        "MNZ060",
                        "MNZ061"
                    ],
                    "SAME": [
                        "027019",
                        "055109",
                        "027085",
                        "027123",
                        "027067",
                        "027171",
                        "027093",
                        "055033",
                        "027163",
                        "055017",
                        "027053",
                        "027003"
                    ]
                },
                "affectedZones": [
                    "https://api.weather.gov/zones/forecast/MNZ068",
                    "https://api.weather.gov/zones/forecast/WIZ023",
                    "https://api.weather.gov/zones/forecast/MNZ066",
                    "https://api.weather.gov/zones/forecast/MNZ062",
                    "https://api.weather.gov/zones/forecast/MNZ057",
                    "https://api.weather.gov/zones/forecast/MNZ059",
                    "https://api.weather.gov/zones/forecast/MNZ058",
                    "https://api.weather.gov/zones/forecast/WIZ025",
                    "https://api.weather.gov/zones/forecast/MNZ063",
                    "https://api.weather.gov/zones/forecast/WIZ027",
                    "https://api.weather.gov/zones/forecast/MNZ060",
                    "https://api.weather.gov/zones/forecast/MNZ061"
                ],
                "references": [
                    {
                        "@id": "https://api.weather.gov/alerts/NWS-IDP-PROD-3934596-3346306",
                        "identifier": "NWS-IDP-PROD-3934596-3346306",
                        "sender": "w-nws.webmaster@noaa.gov",
                        "sent": "2019-11-29T12:09:00-06:00"
                    },
                    {
                        "@id": "https://api.weather.gov/alerts/NWS-IDP-PROD-3934596-3346307",
                        "identifier": "NWS-IDP-PROD-3934596-3346307",
                        "sender": "w-nws.webmaster@noaa.gov",
                        "sent": "2019-11-29T12:09:00-06:00"
                    }
                ],
                "sent": "2019-11-29T16:45:00-06:00",
                "effective": "2019-11-29T16:45:00-06:00",
                "onset": "2019-11-29T21:00:00-06:00",
                "expires": "2019-11-30T03:00:00-06:00",
                "ends": "2019-12-01T12:00:00-06:00",
                "status": "Actual",
                "messageType": "Update",
                "category": "Met",
                "severity": "Severe",
                "certainty": "Likely",
                "urgency": "Expected",
                "event": "Winter Storm Warning",
                "sender": "w-nws.webmaster@noaa.gov",
                "senderName": "NWS Twin Cities/Chanhassen MN",
                "headline": "Winter Storm Warning issued November 29 at 4:45PM CST until December 1 at 12:00PM CST by NWS Twin Cities/Chanhassen MN",
                "description": "...HEAVY SNOW EXPECTED THIS WEEKEND ACROSS CENTRAL MINNESOTA AND\nNORTHWEST WISCONSIN WITH A WINTRY MIX TO THE SOUTH...\n...VERY TIGHT SNOWFALL GRADIENT POSSIBLE FROM CANBY TO THE TWIN\nCITIES AND EAU CLAIRE AREAS SATURDAY...\n\n.A Winter Storm Warning is in effect tonight through Sunday\nmorning for areas along and north of a line from Canby and the\nTwin Cities to Eau Claire. Snow is expected to start this\nevening. A total of 8 to 14 inches are expected by Sunday\nmorning, with the lighter amounts on the southern edge of the\nwarning. From Canby to the Twin Cities and Eau Claire, snow will\nturn to a wintry mix Saturday and then turn back to snow Saturday\nnight and Sunday morning. There is expected to be a very tight\nsnow gradient somewhere near this line with all snow to the north\nand a prolonged mix to the south. Gusty east winds of 30 to 40\nmph could lead to patchy blowing snow as well.\n\nA Winter Weather Advisory is in effect for south central\nMinnesota and portions of west central Wisconsin tonight. The snow\nwill likely turn to rain and possibly a rain/sleet mix Saturday\nmorning before turning back to snow Saturday night.\n\n* WHAT...Heavy mixed precipitation expected. Total snow\naccumulations of 6 to 10 inches and ice accumulations of a\nlight glaze. Winds gusting as high as 40 mph.\n\n* WHERE...Portions of west central Wisconsin and central and\neast central Minnesota.\n\n* WHEN...From 9 PM this evening to noon CST Sunday.\n\n* IMPACTS...Travel could be very difficult. Patchy blowing snow\ncould significantly reduce visibility.\n\n* ADDITIONAL DETAILS...Snow will fall tonight, then turn to a\nwintry mix Saturday before turning back to snow Saturday night.\nPrecipitation may remain all snow north of a line from Canby, to\nthe Twin Cities, and Eau Claire. If this occurs, around a foot\nof snow can be expected.",
                "instruction": "If you must travel, keep an extra flashlight, food, and water in\nyour vehicle in case of an emergency.\n\nThe latest road conditions for the state you are calling from can\nbe obtained by calling 5 1 1.",
                "response": "Prepare",
                "parameters": {
                    "NWSheadline": [
                        "WINTER STORM WARNING REMAINS IN EFFECT FROM 9 PM THIS EVENING TO NOON CST SUNDAY"
                    ],
                    "HazardType": [
                        "Heavy Mixed precipitation, Mixed Precipitation"
                    ],
                    "VTEC": [
                        "/O.CON.KMPX.WS.W.0012.191130T0300Z-191201T1800Z/"
                    ],
                    "EAS-ORG": [
                        "WXR"
                    ],
                    "PIL": [
                        "MPXWSWMPX"
                    ],
                    "BLOCKCHANNEL": [
                        "CMAS",
                        "EAS",
                        "NWEM"
                    ],
                    "eventEndingTime": [
                        "2019-12-01T12:00:00-06:00"
                    ]
                }
            }
        }
    ],
    "title": "current watches, warnings, and advisories for Hennepin (MNZ060) MN",
    "updated": "2019-11-29T22:46:14+00:00"
}
nmlorg commented 4 years ago

UI-wise, the current Minneapolis warning might look like:

Event Title TOMORROW, Sat 30ᵗʰ, 8–11:59ᵖᵐ @ Event Location ⚠ Winter Storm Warning: Heavy Mixed precipitation, Mixed Precipitation Second Event ² ᵈᵃʸˢ Sun, Dec 1ˢᵗ, 5–9ᵖᵐ @ Second Location ⚠ Winter Storm Warning: Heavy Mixed precipitation, Mixed Precipitation

where the hazard line links to either a location-specific URL, like https://forecast.weather.gov/showsigwx.php?warnzone=MNZ060&warncounty=MNC053&firewxzone=MNZ060, or to an advisory-specific URL, like https://alerts-v2.weather.gov/products/NWS-IDP-PROD-3935103-3346792.

nmlorg commented 4 years ago

As of dddc581, the bot displays the temperature and short form of the hourly forecast for each event's start time iff the latter contains one of: 'blizzard', 'freezing', 'heavy', 'hurricane', 'ice', 'snow', 'tornado', 'tropical'.

For weather advisories, I'm leaning toward having the bot announce them as they are posted/are updated/expire, as completely independent messages (even if no regular announcements are configured?).

This could be kept separate:  If a group is configured either with an explicit location, or has explicitly opted in to weather advisories, go ahead.

Otherwise, it could guess at the group's location by examining its configured calendars for upcoming (and also past?) events. It may need to go beyond the currently configured /events range, however, and/or go before the range to check past events, so this might end up being a bit invasive. (m.u.eventutil.get_group_events might need to bypass m.c.multicalendar.MultiCalendar.get_overlap (or partially reimplement it, or call it multiple times) to get a usable collection of events, then perform/duplicate the date range filtering itself.)

This could also be hybrid:  Only an explicitly configured location is used for advisories, but as part of a pseudo schema change the bot could do a one-off examination of each group's configured calendars to set an initial explicit location.

I think I'm leaning toward having the bot self configure on the fly, just calling get_overlap(now, now + max(period, 90 * 24 * 60 * 60)), then manually filtering the returned value for the actual period (the same way it manually filters for count). The return type could then be changed from a list of events to a tuple of (alerts, events). The /events handler could ignore the alerts (maybe?) but the daily announcement loop could examine all groups with one or more calendars (and a timezone?) set and use the existing get_group_events calls to independently identify weather alerts to announce (shoving them into the same cache as the most-recent daily announcement, so it can only post on change).

Raeitus commented 4 years ago

Would be nice to change the weather provider to https://openweathermap.org/ instead of the US Gov Weather as it covers weather world wide and has a lot of APIs to help developers.

It is also used in many open sourced weather apps.

nmlorg commented 4 years ago

Minor implementation note:  We'll always be going from free-form[ish] address string to either alerts or forecast, but won't always (ever?) be pulling both alerts and forecasts at the same time. It might make sense to introduce a new geoutil.Geo (or Location, Point) class with get_forecast(when) and get_alerts() methods.

Alternatively, just change hourlyforecast to accept an address and perform the [self-caching] geocode call (and do the time filtering) itself.

nmlorg commented 4 years ago

Test running 04b9d91 for the past few days, it turns out there's a lot of noise due to both (1) api.weather.gov's intermittent unavailability and (2) NWS frequently updating alerts to tweak the wording, estimates, or affected areas. The latter sometimes causes alerts to be re-posted even if the 250-character blurb doesn't change, simply because the way they update an alert is to cancel it and issue a brand new one in its place (so the alert id, which is included in the URL, changes).

For example:

1:00:04 PM
Winter Weather Advisory NWS-IDP-PROD-3947931-3356042

...QUICK BURST OF HEAVY SNOW TONIGHT THROUGH MONDAY...

.A Winter Storm Warning has been issued for central Minnesota, namely Douglas, Todd, Morrison, Mille Lacs and Kanabec counties, for tonight through Monday morning.

A Winter Weather Advisory re…
3:00:08 PM
Winter Weather Advisory NWS-IDP-PROD-3948131-3356175

...QUICK BURST OF HEAVY SNOW TONIGHT THROUGH MONDAY...

.A Winter Storm Warning has been issued for central Minnesota, namely Douglas, Todd, Morrison, Mille Lacs and Kanabec counties, for tonight through Monday morning.

A Winter Weather Advisory re…
6:00:08 PM
Winter Weather Advisory NWS-IDP-PROD-3948271-3356260

...QUICK BURST OF HEAVY SNOW TONIGHT THROUGH MONDAY WILL IMPACT MONDAY MORNING COMMUTE...

.A Winter Storm Warning continues for central Minnesota, namely Douglas, Todd, Morrison, Mille Lacs and Kanabec counties, for tonight through Monday morning.
…
1:00:06 AM
Winter Weather Advisory NWS-IDP-PROD-3948605-3356489

...QUICK BURST OF HEAVY SNOW WILL IMPACT MONDAY MORNING COMMUTE...

.A Winter Storm Warning continues for central Minnesota, namely Douglas, Todd, Morrison, Mille Lacs and Kanabec counties, through Monday morning.

A Winter Weather Advisory is now i…

A followup should probably include some kind of diffing behavior, possibly literally calling _quick_diff, but tweaking it to at least show 250 characters of context, and possibly allowing it to actually find multiple changes. (Maybe just run it as is on each paragraph?)

nmlorg commented 4 years ago

Feedback:

While it's very considerate of you to post these, my phone, my computer, the radio, and the TV all consistently update me on the weather so it's a little excessive.

These weather updates would be most salient if they were made during or right before any sanctioned events on the calendar

(This particular group has had 11 alerts posted in the past 4 days.)