New windy thresholds?

[ajstewart]

The thread here that popped up when the weather was broken a few days ago: https://old.reddit.com/r/TheSilphRoad/comments/aks0k7/weather_gone_germany/

Suggests a windy threshold of:

Windy Override means that sufficient wind (wind speed over 24.1 km/h or wind gust speed over 31.5) and regardless of what the forecast is, it'll be windy in game. If it's a precipitation type (rain, snow) it'll never be windy, but you'll see the wind effect.

That post lead me down a rabbit hole of attempting to predict our weather and a google search lead me to your bot! Nice work!

[5310]

Thank you! And also thanks for the new resource.

Since DST started our instance of this bot has been pretty inaccurate, which makes us suspect that the forecast hours might have changed, despite us thinking the forecast hours were local to the timezone. I've been trying out some new models, and this'll help.

It's just that we get wind very very rarely around here, so I'll be a while before I can be sure of any new wind models.

[ajstewart]

No problem, that excellent resource put me on this path (there are some other comments in there about rain as well).

We get windy a lot here, I could test out the wind thresholding suggested in that thread. I'd probably need a little help coding it though, still working my way around js.

I'm currently in every hour mode to try and nail down our weather pull times.

[5310]

I actually just pushed th0rnleaf's wind model a while ago https://github.com/5310/discord-bot-castform/commit/8d317e22406d876b8a093a0279cd0680ac3b2c0b#diff-a66005028d1762c28eeaeaedfb75fbd5R71 !

Apparently, the old model was predicting (with the wrong pull hours, and no corroboration in-game) that one of our cells would get superficial winds. Hopefully it picks up later in the week.

On Thu, Jan 31, 2019 at 11:21 AM Adam Stewart notifications@github.com wrote:

No problem, that excellent resource put me on this path (there are some other comments in there about rain as well).

We get windy a lot here, I could test out the wind thresholding suggested in that thread. I'd probably need a little help coding it though, still working my way around js.

I'm currently in every hour mode to try and nail down our weather pull times.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/5310/discord-bot-castform/issues/2#issuecomment-459224794, or mute the thread https://github.com/notifications/unsubscribe-auth/AAG5EjOJQt6MeF0wpkWW7pCMLn5PvtXCks5vIoRzgaJpZM4abVXu .

[ajstewart]

Ah cool, ok my js is good enough that I matched your commit with my own change.

We actually have windy weather now! I will keep monitoring it.

[5310]

Lucky you. And thanks again.

On Thu, Jan 31, 2019 at 12:27 PM Adam Stewart notifications@github.com wrote:

Ah cool, ok my js is good enough that I matched your commit with my own change.

We actually have windy weather now! I will keep monitoring it.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/5310/discord-bot-castform/issues/2#issuecomment-459237247, or mute the thread https://github.com/notifications/unsubscribe-auth/AAG5Et7kpb7sSSKbrtHrzqWoMT_BobDBks5vIpPFgaJpZM4abVXu .

[ajstewart]

Looking at your code and the model proposed by th0rnleaf, am I right in thinking that currently wind and rain override is possibly being applied to too many accuweather labels?

E.g. I just had a forecast of mostly cloudy w/ thunderstorms but the bot has interpreted as rain. Though th0rnleaf model suggests this should be approached as cloudy (of course ignoring that the model might not be accurate, seems to be the best there is though).

[5310]

Are you sure your current forecast comes from the most recent pull hour? Or at least hasn't changed since the last pull hour?

When our instance of the bot was still running back in summer other than superficial rain and wind we actually never had it fail for at least the first two or three months the members were submitting observations. Which is to say, simple maps from AW forecast label to PoGO label was always correct, (and we get a lot of rain in summer too.) Up until the time we were able to decide on our pull hour we had a lot of suspicious false positive hours where most of them matched but then one hour out of a week would maybe diverge like it does in your case.

Back in summer our pull hours were (fo' sho') midnight, 8am, noon, and 4pm. Say at 1pm the current fresh forecast for 2pm was cloudy, but it was actually rainy back on the noon forecast, we'd still get rain at 2.

[ajstewart]

Sorry I didn’t explain properly - it almost certainly isn’t the pull hour, I’m purely talking about how it interprets what it sees if it was the pull hour. So mostlycloudy with thunderstorm apparently should be cloudy according to that new model with no rain or wind override. That’s what I mean, I’m talking about the PoGo model regardless if it’s the pull hour or not.

On 31 Jan 2019, at 18:49, Sayantan Chaudhuri notifications@github.com wrote:

Are you sure your current forecast comes from the most recent pull hour? Or at least hasn't changed since the last pull hour?

When our instance of the bot was still running back in summer other than superficial rain and wind we actually never had it fail for at least the first two or three months the members were submitting observations. Which is to say, simple maps from AW forecast label to PoGO label was always correct, (and we get a lot of rain in summer too.) Up until the time we were able to decide on our pull hour we had a lot of suspicious false positive hours where most of them matched but then one hour out of a week would maybe diverge like it does in your case.

Back in summer our pull hours were (fo' sho') midnight, 8am, noon, and 4pm. Say at 1pm the current fresh forecast for 2pm was cloudy, but it was actually rainy back on the noon forecast, we'd still get rain at 2.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[5310]

Ah, then it must indeed have been an override. The model the bot follows considers rain snow fog and wind to all act as overrides if extreme enough. In this case, the chance of rain might've been above 60%.

[5310]

Perhaps I should consider testing a more simplified model. The code is modular enough.

[ajstewart]

I would try following the model in his spreadsheet (https://docs.google.com/spreadsheets/d/1v51qbI1egh6eBTk-NTaRy3Qlx2Y2v9kDYqmvHlmntJE/edit#gid=0) so the code would need some sort of filter on if the AccuWeather label is non-override one (for example = mostlycloudywtstorms) then do not apply wind or rain override.

I have a candidate pull hour that is still going strong and the only hour it got it wrong was the one above, but it would have been correct following th0rnleaf model, in that case cloudy, which it was in-game.

[5310]

I'd actually already started implementing th0rnleaf's model from scratch before decided to save time by making the frankenmodel... I'll split them apart later today and ping this thread.

[5310]

(Glitch might have a glitch with commits containing apostrophe's...)

I've pushed th0rnleaf's model here: https://github.com/5310/discord-bot-castform/blob/6df2aadbb13d6e01eefbceef6391e6efb435eed0/pogo-th0rnleaf.js

[ajstewart]

Cool thanks for doing it so fast! I'll pull this now ready for the next hour. Still a promising run going either 3pm or 4pm local time forecast when you apply the th0rnleaf model, which is around the time you mentioned interestingly enough. Just to note I'm also pulling the forecasts at 15 past the hour.

I'll report back with how I get on.

[5310]

I for one hope it's not 4pm. Our 4pms don't match any of the first three hours and sometimes even later hours too! :D

On Thu, Jan 31, 2019 at 4:59 PM Adam Stewart notifications@github.com wrote:

Cool thanks for doing it so fast! I'll pull this now ready for the next hour. Still a promising run going either 3pm or 4pm local time forecast when you apply the th0rnleaf model, which is around the time you mentioned interestingly enough. Just to note I'm also pulling the forecasts at 15 past the hour.

I'll report back with how I get on.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/5310/discord-bot-castform/issues/2#issuecomment-459311123, or mute the thread https://github.com/notifications/unsubscribe-auth/AAG5EqLFrCPEpqqQgFoPi_5VUAu99PIKks5vItN-gaJpZM4abVXu .

[5310]

I went ahead and added in superficial effects following th0rnleaf's writeup. The dominant weather effect condition hasn't changed, so it's safe to update if you want: https://github.com/5310/discord-bot-castform/blob/85193b357eade10f171419facfcbb6ec053e0566/pogo-th0rnleaf.js

[ajstewart]

In the latest version the model is not implemented correctly. I didn't notice it last night but you're missing defining the windy allowed list and also things are becoming windy that shouldn't be, which I'm not actually sure why that is happening.

Take these two examples:

            {
                "epoch": 1548997200,
                "querydate": "2019-02-01",
                "queryhour": "07",
                "date": "2019-02-01",
                "hour": "16",
                "forecast": {
                    "label": "mostlycloudy",
                    "temp": 22.5,
                    "visib": 11.3,
                    "wind": 35.2,
                    "gust": 38.9,
                    "cloud": 85,
                    "precip": 48,
                    "rain": 48,
                    "snow": 0,
                    "humid": 75,
                    "dew": 17.9,
                    "ceiling": 0.61,
                    "light": true,
                    "uv": 2
                },
                "prediction": {
                    "dominant": "cloudy",
                    "superficial": {
                        "windy": true
                    }
                }
            },
            {
                "epoch": 1549000800,
                "querydate": "2019-02-01",
                "queryhour": "07",
                "date": "2019-02-01",
                "hour": "17",
                "forecast": {
                    "label": "showers",
                    "temp": 22.1,
                    "visib": 8,
                    "wind": 33.3,
                    "gust": 37,
                    "cloud": 85,
                    "precip": 62,
                    "rain": 62,
                    "snow": 0,
                    "humid": 77,
                    "dew": 18,
                    "ceiling": 0.61,
                    "light": true,
                    "uv": 1
                },
                "prediction": {
                    "dominant": "windy",
                    "superficial": {
                        "windy": true
                    }
                }

In the first case we have mostlycloudy with wind above the threshold. mostlycloudy is a label where wind override should be allowed and the dominant prediction should be windy.

In the second case the label is showers. This is not on the windy override list and the dominant prediction should be rain.

[ajstewart]

I think I see what's going on: https://github.com/5310/discord-bot-castform/blob/85193b357eade10f171419facfcbb6ec053e0566/pogo-th0rnleaf.js#L68

Showers has the label rain in the dominant list so showers can become windy.

[ajstewart]

Sorry third message! I reintroduced the windyList and then made the windyable check just a simple check on this list:

const windyable = windyList.includes(label)

see my branch: https://github.com/sydney-pogo-admin/discord-bot-castform/blob/sydney/pogo-th0rnleaf.js.

Now my forecasts dominants are consistent with the model - I believe.

p.s. this was easy for me to see as we are predicted windy all day! 😆

[5310]

Oh, oops! My lazy solution was a bit too lazy it seems ^^;

I'm going to make windyable a part of the main map again.

(Also, I'm not at all jealous at your all-day wind. Nope.)

[5310]

Re-fixed: https://github.com/5310/discord-bot-castform/commit/7ef6c1292c6443a5ba8b38ffc27100eeee681889#diff-1c7a17351d0ce1d62f1fb9f899c637d2

Btw, what pull hours did you settle at? I think our most confident guess is now 06, 14, 18.

[ajstewart]

Great thanks!

I haven't been able to work out the hours today, the hourly forecasts have been too static. But the model has now been spot on since I made the edits:

I'll have to wait for changes. Monitoring overnight would help as well 😛

[5310]

TWIMC: We're currently working with the th0rnleaf model and testing to see what the pull hours are and how they change. Juicy discoveries will be posted :]

[gobalopper]

I believe the windy > 24.1 thing might be accurate. We had a forecast of 24.1km/hr today at 11AM and the in-game weather didn't change. However, it was forecast to be 25.9km/hr an hour later at 12PM and the in-game weather did change to windy at that point. Perhaps Niantic is using 15mph?

[ajstewart]

This recent thread on the Silph Road is what I also believe is nearest to what is actually going on https://old.reddit.com/r/TheSilphRoad/comments/bfbwy4/windy_weather_requirements_may_have_been_changed/.

The conclusion of or just gust speeds >30 km/h is pretty much what I came to independently, and windy weather triggering did seem to change at some point in the past few months. It became not as predictable as it was before. I'm in Sydney, Australia and the 24.1 has never worked for me (I almost start to suspect that some local information may be incorporated somehow into the model). Gusts at 29.6 seem to be a magic number for me but it's difficult to work out if it should be >= or > and there may be some lower threshold on the wind speed to go along with this.

I'm currently running wind > 20 && gust >= 29.6 which seems to get it pretty much spot on for us. Just a little unpredictable when the speeds are right around those thresholds (which tells me it's not exactly right but we at least get an idea that it could be windy when it's right at the "windy border").

[5310]

We don't get a lot of windy here at all, but since we moved to @ajstewart's wind > 20 && gust >= 29.6 we've never had any false positives or negatives regarding wind, and a couple of hours' of hits (excepting a few identical forecasts missing because of the omnipresent mystery of the pull hours.)

This is a good reminder that I have to update this repository with the new model though. BRB— Updated!

[gobalopper]

I'll update then. :D Thanks!

[SwampThingTom]

@5310 Minor but your comment above says the threshold is wind > 20 && gust >= 29.6 but the code appears to be using wind > 20 && gust > 30.

[ajstewart]

Yeah I think that was a difference between us in tinkering with the threshold. I found that 29.6 was always a special number so I used the >= but @5310 in his area used >30.

I haven't done any further investigating but as the wind speeds come in discrete values the difference is small, basically wind will start for 5310 at one gust notch above what mine is set to. The answer might be to see what works for you.

[cy11100]

Hi, thanks for this program! I've been running it for a couple of days and it does work very well when the correct local weather code is selected. I've been using a location as close as possible to the middle of the S2 L10 cell for the location, which predicts weather conditions with >95% accuracy so far.

Following is based on commit adcef34 from v2-webhook branch. I've only recently restarted playing so my understanding of the system may be incomplete, so please do excuse any errors that I make below.

I have a number of recent datapoints that suggest the windy predictor using the default model-ajstewart.js may be inaccurate, this could have been due to a potential lack of coverage of this specific wind speed state in previous tests, a change on Niantic's end, or possibly a local difference in the conversion from API returns to ingame condition (less likely).

I have read through the previous discussions on windy thresholds https://github.com/5310/discord-bot-castform/issues/2 and https://github.com/5310/discord-bot-castform/issues/20 , although neither discusses this specific case. I also found this previous weather prediction converter from th0rnleaf on /r/thesilphroad using Accuweather API, which suggests that the operator should be || instead of &&. I assume that is the origin of model-th0rnleaf.js.

The correct and incorrect predictions are listed below, I have also included the raw JSON from the datetime in question. If you have any previous datapoints that pointed towards the && variant, it would be nice to have these as well so as to create a better predictor model.

Incorrect predicted weather using model-ajstewart.js, differs from ingame weather:

• 13: Sunny
• 14: Sunny
• 15: Sunny
• 16: Sunny
• 17: Sunny

Correct predicted weather using modified model-ajstewart.js. Matches ingame weather. Obtained by changing : const windy = wind > thresholds.dominant.wind && gust > thresholds.dominant.gust to const windy = wind > thresholds.dominant.wind || gust > thresholds.dominant.gust

• 13: Sunny
• 14: Windy
• 15: Windy
• 16: Windy
• 17: Sunny

Raw json for the above timings: (click to expand)

``` { "epoch": 1692680400, "querydate": "2023-08-22", "queryhour": "10", "date": "2023-08-22", "hour": "13", "label": "mostlysunny", "temp": 30.9, "visib": 16.1, "wind": 14.8, "gust": 29.6, "cloud": 26, "precip": 3, "rain": 3, "snow": 0, "humid": 67, "dew": 24.2, "ceiling": 9.144, "light": true, "uv": 11 }, { "epoch": 1692684000, "querydate": "2023-08-22", "queryhour": "10", "date": "2023-08-22", "hour": "14", "label": "mostlysunny", "temp": 30.9, "visib": 16.1, "wind": 14.8, "gust": 31.5, "cloud": 26, "precip": 3, "rain": 3, "snow": 0, "humid": 65, "dew": 23.8, "ceiling": 9.144, "light": true, "uv": 7 }, { "epoch": 1692687600, "querydate": "2023-08-22", "queryhour": "10", "date": "2023-08-22", "hour": "15", "label": "mostlysunny", "temp": 31.4, "visib": 16.1, "wind": 14.8, "gust": 33.3, "cloud": 26, "precip": 3, "rain": 3, "snow": 0, "humid": 62, "dew": 23.4, "ceiling": 9.144, "light": true, "uv": 4 }, { "epoch": 1692691200, "querydate": "2023-08-22", "queryhour": "10", "date": "2023-08-22", "hour": "16", "label": "mostlysunny", "temp": 30.7, "visib": 16.1, "wind": 14.8, "gust": 31.5, "cloud": 26, "precip": 3, "rain": 3, "snow": 0, "humid": 63, "dew": 23.1, "ceiling": 9.144, "light": true, "uv": 2 }, { "epoch": 1692694800, "querydate": "2023-08-22", "queryhour": "10", "date": "2023-08-22", "hour": "17", "label": "mostlysunny", "temp": 30.2, "visib": 16.1, "wind": 14.8, "gust": 29.6, "cloud": 26, "precip": 3, "rain": 3, "snow": 0, "humid": 64, "dew": 22.9, "ceiling": 9.144, "light": true, "uv": 1 },

[ajstewart]

Hi @cy11100, this thread brings back memories 😄

It's entirely possible things have changed, there are hints recently that something changed (e.g. https://old.reddit.com/r/TheSilphRoad/comments/13dsz56/ingameweather_doesnt_match_accuweather/).

I spent a while back then monitoring weather and definitely the wind thresholds in the original th0rnleaf were not working for my location (Sydney at the time), I remember speaking to th0rnleaf at the time about them.

However, yeah this was around 5 years ago now and I barely play the game anymore and no longer live in the same area to monitor. I did get reports from people in Sydney in the past 6 months that the windy weather was starting to deviate from the predictions, but I have not investigated any further.

I always suspected that the thresholds could be derived from some sort of local or historical wind metrics but again never really followed this up. All those years ago I was speaking to a few people in different locations and it seemed really difficult to line up a model to fit all.

From your analysis it seems like the original model is closer to what you're seeing, might be worth switching back the default, but I'd always advise to do what you've done and see what works for you and adjust. While I may have missed something in the last few years, I don't think the windy threshold have ever been fully understood!

[cy11100]

Thanks for the thoughtful reply! I personally don't have time for it right now, but one possible solution for a data-driven global prediction model would be using the (undocumented, but easily reverse engineered with F12 web tools) local scanner APIs (such as sydneypogomap.com) and then automatically checking the in-game weather from live scanner data against the predictions. This same scanner code (and therefore API) appears to be used in multiple locations such as London, NYC, Vancouver, Singapore, so this would work for creating a global model.

I suspect that much of the mismatches may come from the S2 L10 cells pulling weather from a different Accuweather location vs the program is retrieving from Accuweather. There doesn't currently seem to be a good way to go from a geolocation (middle of S2 L10 cell) to Accuweather location code, and that could account for a large proportion of the inaccuracies.

I think most people in this thread have probably moved on from the game due to life changes, but it would definitely be an interesting way to resolve this situation, assuming the scanner operators don't get shut down due to Niantic changes or their own reasons.