Forecast seems 30 mins out

[peterdudfield]

The forecaster always looks about 30 mins off

By looking at the gsp data at 21:55, the batch has the following values 19:30, 20:00, 20:30, 21:00, 21:30 and the model uses the first 4 values.

In training the values it has taken are 19:30, 20:00, 20:30, 21:00, 21:30.

Out of date This means in the morning, i.e 9.15, the live model uses times: 7:00, 7:30, 8:00 and 8:30, tried to predict for 9:00 and the training has used times: 7:30, 8:00, 8:30, 9:00, and tried to predict for 9:30.

More to follow

[peterdudfield]

The training data, t0=05.35 gsp times are '2020-09-04T03:30:00.000000000', '2020-09-04T04:00:00.000000000', '2020-09-04T04:30:00.000000000', '2020-09-04T05:00:00.000000000', '2020-09-04T05:30:00.000000000', '2020-09-04T06:00:00.000000000', '2020-09-04T06:30:00.000000000', '2020-09-04T07:00:00.000000000', '2020-09-04T07:30:00.000000000', '2020-09-04T08:00:00.000000000', '2020-09-04T08:30:00.000000000', '2020-09-04T09:00:00.000000000', '2020-09-04T09:30:00.000000000', '2020-09-04T10:00:00.000000000', '2020-09-04T10:30:00.000000000', '2020-09-04T11:00:00.000000000', '2020-09-04T11:30:00.000000000', '2020-09-04T12:00:00.000000000', '2020-09-04T12:30:00.000000000', '2020-09-04T13:00:00.000000000',

[peterdudfield]

Currently, that model is trained on 30 mins to 0 minutes of historic satellite data, Live used 1 hour but due to delay uses 1 hour to 30 mins historic values

Trained 05.35 uses '2020-09-04T05:05:00.000000000', '2020-09-04T05:10:00.000000000', '2020-09-04T05:15:00.000000000', '2020-09-04T05:20:00.000000000', '2020-09-04T05:25:00.000000000', '2020-09-04T05:30:00.000000000', '2020-09-04T05:35:00.000000000'.

   Infact in live nothing before 05:00 is available. Need to put in delay of 1 hour

[peterdudfield]

I think i solved it.

This happened becasue

• batch.t0 was set to rounded down 30 mins + 1 second.
• first target_time = batch.metadata.t0_datetime_utc + 30 minutes Problem: When in last 5 mins, the batch is made at 12:57:01, which means the t0 is 13:00, so we predict for 13:30. But we should be predicting for 13:00. We then assign the 13:30 at 13:00. This gives an over prediction in the morning, and under predictin in the afternoon

Now

• we have rounded the batch.t0 down to 5 mins. This gets rid of this problem.
• first target_time = ceil(batch.t0, 30T), but add 30 mins when we are in the first 12:30 to 12:35, then we add 30 mins.