🐛 Weather determination does not match retail; potentially deterministic instead of random choice

almuth150 commented 6 months ago

I affirm:

[x] I understand that if I do not agree to the following points by completing the checkboxes my issue will be ignored.
[x] I have read and understood the Contributing Guide and the Code of Conduct.
[x] I have searched existing issues to see if the issue has already been opened, and I have checked the commit log to see if the issue has been resolved since my server was last updated.

OS / platform the server is running (if known)

All

Branch affected by issue

base

Steps to reproduce

Visit various areas and note that weather does not match retail. Current LSB implementation uses an 8 in-game year weather cycle with patterns for each individual zone, without regards to simultaneous weather types occurring in linked weather pattern zones.

Expected behavior

Looking at some previous discussions on the FFXI private server discord reverse engineering room, apparently weather on all retail servers occur at the same time, and the pattern is potentially deterministic rather randomly determined from a selection of 3 possibilities (ie. the same weather happens in the same area at the same time in the weather cycle, every time). The cycle is said to last 6 in-game years, or 86.4 IRL days (ie. 7464960 seconds).

Firstly, anyone with a retail account can help with an addon obtained here: https://github.com/zach2good/WeatherReporter. Runs passively in the background and sends data automatically while playing on retail. More data is always appreciated.

Various indoor areas have their weather linked to their connecting associated outdoor area - this page for weather reporters is a good start for determining linked areas: https://ffxiclopedia.fandom.com/wiki/Weather_Reporter. However, it is not perfect. Example, it lists Rabao and Altepa Desert areas separately, but when I talk to the appropriate weather reporter NPCs (on a LSB server mind you), I only get the option for Altepa Desert. Going by that, are some outdoor areas linked as well? Example, the two Riverne sites with each other (these are technically dungeons I suppose), Lufaise Meadows and Misareaux Coast, the pair of starter zones outside each main city + their neighbouring dungeon areas (heck, are they also linked to their main city???), and then the two Altepa deserts with Rabao and the indoor Quicksand Caves.

Because of the above, weather should match exactly across the linked areas. Determining which areas are linked would help with the construction of weather patterns, since data for the linked zones would be combined for more data points in the area.

Related issues that would be resolved with a more accurate weather implementation: https://github.com/LandSandBoat/server/issues/410 https://github.com/LandSandBoat/server/issues/704

cocosolos commented 6 months ago

Link to the current sqlite db for the weather reporter (source)

You can open this with something like this. Here's a query to look at a zones data (244 Upper Jeuno in this example), grouped into when they happen in the proposed 6-year cycle, starting from the SE epoch:

select 
    count(*) as datapoints,
    name,
    (timestamp - 1009810800) % 7464960 / 10*10 as cycleTime 
from 
    weather_data 
left join 
    weather_names 
on 
    weather_names.weather = weather_data.weather 
where 
    zone = 244 
group by 
    cycleTime, name;

This rounds them down to the 10 second mark since there seems to be a small variance probably due to network latency or something.

This one should show when there are matching cycle times in the same zone but different weather, something that shouldn't generally happen if it's fully deterministic. It looks at all zones and currently only has a few results that might indicate fog and sunshine are a random chance instead of None, but more info is needed.

WITH CycleTimeCounts AS (
    SELECT
        (timestamp - 1009810800) % 7464960 AS cycleTime,
        COUNT(DISTINCT name) AS nameCount
    FROM weather_data
    LEFT JOIN weather_names ON weather_names.weather = weather_data.weather
    GROUP BY cycleTime, zone
)
SELECT
    *,
    name,
    (timestamp - 1009810800) % 7464960 AS cycleTime
FROM weather_data
LEFT JOIN weather_names ON weather_names.weather = weather_data.weather
WHERE (timestamp - 1009810800) % 7464960 IN (
    SELECT cycleTime FROM CycleTimeCounts WHERE nameCount > 1
)
GROUP BY cycleTime, name;

That first query could be used to compare different zones, and if it's likely they share the same pattern then the data from both zones could be combined to improve the dataset.

zach2good commented 6 months ago

Related: some zones don't have 'none' weather, so they need this improvement https://github.com/LandSandBoat/server/pull/5831

almuth150 commented 6 months ago

Don't know if I should make a separate issue for this, but I noticed that Jugner Forest [S] weather is a copy and paste of Vunkerl Inlet [S], resulting it in having weather that it's not supposed to have. It's not supposed to have any wind weather, nor double thunder weather.

That would be resolved by resolving this issue, of course, but I wonder if a temporary measure would be appropriate. Like what was done with the northlands [S] areas, using the weather patterns of their present versions.

LandSandBoat / server