openclimatefix / pv-pseudo-experiments

Repo containing the training and experimentation code for PV MetNet and Pseudo-Irradiance models
MIT License
2 stars 0 forks source link

Aerosol Data And Analog Ensembles #2

Open jacobbieker opened 1 year ago

jacobbieker commented 1 year ago

We want to determine if it is worth including aerosol data in our experiments and forecasts. This issue is just recording some of the literature and ideas about how to use it.

Arxiv/Blog/Paper Link

https://www.researchgate.net/profile/Miguel-Prosper/publication/360978841_Development_of_a_solar_energy_forecasting_system_for_two_real_solar_plants_based_on_WRF_Solar_with_aerosol_input_and_a_solar_plant_model/links/6297124fc660ab61f85695e7/Development-of-a-solar-energy-forecasting-system-for-two-real-solar-plants-based-on-WRF-Solar-with-aerosol-input-and-a-solar-plant-model.pdf

Detailed Description

This paper uses ECMWF CAMS aerosol forecast over India and southern Spain to determine the effects of using it with GFS forecast data. It has a ablation study on how useful the aerosols were, and found a ~10.9% MAE improvement vs just GFS solar information, and had more realistic split between diffuse and direct solar radiation on days with high aerosol forecasts, as well as a 9% NMAE improvement on forecasting temperature. Both underestimated the amount of clouds (which hopefully including satellite cloud imagery would help fix.)

Context

This paper seems like a fairly large improvement including the aerosol forecast.

jacobbieker commented 1 year ago

This paper has a simple MLP improve RMSE from 38.24 to 37.78 when incorporating CAMS forecast, and down to 32.75 when also including AERONET and an angstrom(?) coefficient to help correct the CAMS forecast. Not entirely sure what the angstrom coefficient is, but still an improvement, although fairly small from just CAMS. These forecasts were done over the Middle East, where there is a lot of dust, and fairly clear skies in general.

jacobbieker commented 1 year ago

This is the AERONET that is mentioned https://aeronet.gsfc.nasa.gov/new_web/download_all_v3_aod.html Updated weekly it seems? Not sure if useful for our forecasting

jacobbieker commented 1 year ago

This project actually seems quite relevant to what we are doing, they built an end-to-end forecasting system for operators in the US. Analog Ensemble gave an improvement of ~17% in RMSE for their power forecasts for 0-72 hours.

jacobbieker commented 1 year ago

@dantravers @JackKelly Still going through and finding things, but some interesting results in the literature so far. Aerosols seem to improve forecasts, and the Sun4Cast project linked above had a lot of moving parts, but interesting results, seems to suggest Analog ensembles improved deterministic and probabilistic forecasts.

jacobbieker commented 1 year ago

This one suggests when including aerosol index, the MAPE decreased by halfish for both sunny and cloudy days

JackKelly commented 1 year ago

This section of the Solar Energy Forecasting book is relevant on aerosols... (if you want to buy your own copy then pls expense it to ocf!)

PXL_20230418_171045382.jpg

PXL_20230418_170959062.jpg

PXL_20230418_171019590.jpg

PXL_20230418_171030291.jpg

And this paper is relevant https://pubs.rsc.org/en/content/articlelanding/2018/ee/c8ee01100a

dantravers commented 1 year ago

Great. Let’s pick it up in the next ML meeting. Thanks for the info, @.> @@.!

From: Jack Kelly @.> Sent: Tuesday, April 18, 2023 6:12 PM To: openclimatefix/pv-pseudo-experiments @.> Cc: dantravers @.>; Mention @.> Subject: Re: [openclimatefix/pv-pseudo-experiments] Aerosol Data And Analog Ensembles (Issue #2)

This section of the Solar Energy Forecasting book is relevant on aerosols...

https://user-images.githubusercontent.com/460756/232853117-31417edb-2673-4700-85b5-4f06a5e3a115.jpg

https://user-images.githubusercontent.com/460756/232853118-70ad1085-3da9-4488-9b1b-6b8d6bff84fe.jpg

https://user-images.githubusercontent.com/460756/232853119-53fe9462-ff80-4ad9-b459-9107266bacbe.jpg

https://user-images.githubusercontent.com/460756/232853120-133ce277-e94b-4617-8bf2-726bd29d36f3.jpg

— Reply to this email directly, view it on GitHub https://github.com/openclimatefix/pv-pseudo-experiments/issues/2#issuecomment-1513522319 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AJGGYOZMHXOTJK4ST5OHLUDXB3DOBANCNFSM6AAAAAAXCXMVQ4 . You are receiving this because you were mentioned.Message ID: @.***>