About input data for model inference

[BigShuiTai]

Hello, can I use GFS analysis data for model inference? If yes, how I can calculate toa_incident_solar_radiation parameter with its dataset?

[alvarosg]

Hi, so my recommendation in this case would be to use the pysolar library to build something as similar as possible to the toa_incident_solar_radiation, integrated for one hour up to the time of the input/target. The model should not be extremely sensitive to this parameter, and perhaps if you can get it to be below 1% difference from that of ERA5, the model is probably going to cope ok with it. You can always check against one of the dates for which ERA5 is available.

Alternatively, you may want ask ECMWF in case they already have some open source code, or can make available some code to compute that feature for any arbitrary date.

We may eventually switch to our own pysolar implementation for computing this, but it is unclear of what the timeline for open sourcing this code this would be.

Hope this helps!

[BigShuiTai]

Hi, so my recommendation in this case would be to use the pysolar library to build something as similar as possible to the toa_incident_solar_radiation, integrated for one hour up to the time of the input/target. The model should not be extremely sensitive to this parameter, and perhaps if you can get it to be below 1% difference from that of ERA5, the model is probably going to cope ok with it. You can always check against one of the dates for which ERA5 is available.

Alternatively, you may want ask ECMWF in case they already have some open source code, or can make available some code to compute that feature for any arbitrary date.

We may eventually switch to our own pysolar implementation for computing this, but it is unclear of what the timeline for open sourcing this code this would be.

Hope this helps!

Thank you very much! I will have a try with your suggestions.

[csubich]

Hi, so my recommendation in this case would be to use the pysolar library to build something as similar as possible to the toa_incident_solar_radiation

To add a comment here, I'm working on this problem myself. pysolar is specialized for the idea of solar power deployments at the bottom of the atmosphere, so it needs some modification to give reasonable top of atmosphere radiation.

Some of the internal routines can give the solar angle at a particular location and time, then I found a Sandia National Labs page describing the extraterrestrial radiation component taking into account the eccentricity of Earth's orbit.

There's still a residual mismatch (of a fraction of a percent) with the ECMWF data, but it seems that IFS/ERA5 may have used a more complicated solar model that takes the solar cycle into account.

As an additional note about the data, toa_incident_solar_radiation is the accumulated solar radiation over the 1h leading up to the specified time. For example, for an analysis or forcing dataset describing 2022-Jan-1 6Z, the solar radiation field would be the integrated radiation from 5Z to 6Z. For a quarter-degree grid, integrating radiation with a 1m timestep over this period gives a smooth field without obvious stair-stepping (0.25 deg x 24h / 360 deg = 1m). Taking an instantaneous radiation figure and multiplying by 3600s (converting W to J) will give a radiation field with the wrong shape.

[ihecha]

Hi, so my recommendation in this case would be to use the pysolar library to build something as similar as possible to the toa_incident_solar_radiation

To add a comment here, I'm working on this problem myself. pysolar is specialized for the idea of solar power deployments at the bottom of the atmosphere, so it needs some modification to give reasonable top of atmosphere radiation.

Some of the internal routines can give the solar angle at a particular location and time, then I found a Sandia National Labs page describing the extraterrestrial radiation component taking into account the eccentricity of Earth's orbit.

There's still a residual mismatch (of a fraction of a percent) with the ECMWF data, but it seems that IFS/ERA5 may have used a more complicated solar model that takes the solar cycle into account.

As an additional note about the data, toa_incident_solar_radiation is the accumulated solar radiation over the 1h leading up to the specified time. For example, for an analysis or forcing dataset describing 2022-Jan-1 6Z, the solar radiation field would be the integrated radiation from 5Z to 6Z. For a quarter-degree grid, integrating radiation with a 1m timestep over this period gives a smooth field without obvious stair-stepping (0.25 deg x 24h / 360 deg = 1m). Taking an instantaneous radiation figure and multiplying by 3600s (converting W to J) will give a radiation field with the wrong shape.

Have you solved this problem? I am troubled by this problem, is there any demo code?