Switch live satellite from 5 to 15 mins

[peterdudfield]

Also we are checking if there are problems when the Sat consumer switches from 5 mins to 15 mins. We want to

1. Turn on Satellite data, and check PVnet runs ok (maybe an hour or so once all data is collected)

2. • switch to 15 mins data for a few hours,
   • check nothing bad happens, not spikes in forecast, forecast is continues to be made
   • then swicth back

2023-07-11

• 10:57, changed Satelilte consumer to get 15 mins data
• 11:00, start sat consumer, data from 10 to 11
• 11:16 Satellite consumer finishes
• 11:17 PVnet forecast started, no logs, so not sure what has been loaded
• 11:23 PVnet finished at had no errors, and no spike in forecasts
• 11:45 PVnet did not run, as there were zeros in the satellite data
• 12.15 PVnet did not run, as there were zeros in the satellite data
• 12.43 the satellite data goes from '2023-07-11 09:15 to '2023-07-11 11:00'
• 12.45 PVnet did not run, as there were zeros in the satellite data
• 13.15 PVnet did not run, as there were zeros in the satellite data
• 13.45 PVnet did not run, as there were zeros in the satellite data
• 14.15 PVnet did not run, as there were zeros in the satellite data
• 14,30 Deploiedy version 2.1.12
• 14.45 PVnet did not run, as there were zeros in the satellite data. In index 0-10. The satellite file datestamps are '2023-07-11T11:00:00.000000000', '2023-07-11T11:15:00.000000000', '2023-07-11T11:30:00.000000000', '2023-07-11T11:45:00.000000000', '2023-07-11T12:00:00.000000000', '2023-07-11T12:15:00.000000000', '2023-07-11T12:30:00.000000000', '2023-07-11T12:45:00.000000000', '2023-07-11T13:00:00.000000000'
• Found out that 42% of nans in each satellite timestep Later
• 16.27 PVnet ran on 5 min data

[peterdudfield]

import xarray as xr d = xr.open_dataset('zip::s3://nowcasting-sat-development/data/latest/latest_15.zarr.zip',engine='zarr')

[peterdudfield]

Unfortunately it looks like the 5 minute satellite data wasnt turned off. This caused it to just laod the 5 minute data

[peterdudfield]

Swicth satellite to 15 minute data, we leave it over night to run and see if it runs ok

[peterdudfield]

Managed to run PVnet locally with 15 minute satellite data with fix above

[peterdudfield]

Version 2.12.13 now runs on 15 minute satellite data

[peterdudfield]

Turned satellte backup off at 2023-07-13 09:15

[dfulu]

@peterdudfield could this be closed now?