PVNet performance calculation

[ryanbradshawcodes]

From the discussion of experiments regarding PVNet, the current best PVNet2.1 model scores a (effective) normalised MAE of 0.02951 (current as of 17 July 2023) on a validation set sampled from 2021.

What is the process for calculating this normalised MAE?

I'm working on my own set of models for an independent project, and my test set is 4 months of data, spread out to be the middle month of each season across 2019. I calculate the MAE by doing the absolute difference between actual PV generation (MW) values and predicted PV generation (MW), then finding the mean. I get the normalised MAE by dividing the MAE by the average actual PV generation (MW) over the test set. Is this the same as is done here for PVNet, or is the MAE given from using the difference between the yield generation (generation/capacity of gsp)?

Could the document be updated to make the calculation clearer, please?

[peterdudfield]

@dfulu would you be to have a look at this?

Thanks

[dfulu]

Hi @ryanbradshawcodes, that document has unfortunately gone somewhat stale. I'll attempt to rewrite it over the coming months. So for the moment, I'm not going to update the document in favour of a proper rewrite later.

We also have a workshop paper that has been accepted at the Tackling Climate Change with Machine Learning at ICLR 2024. It has a lot of the experimental details, and should be openly available online in the next month or two.

With regards to your immediate question, we normalise the MAE by dividing by the effective capacity of the GSPs as estimated by PVLive. The effective capacity takes into account that some PV panels inside a GSP break or just that their efficiency decays with time, so the effective capacity is always lower than the installed capacity which is just the sum of PV panels which have been installed within a GSP