search

alexander-petkov / wfas

A placeholder for the WFAS project.
4 stars 1 forks source link

Explore GEOS forecast products #33

Open alexander-petkov opened 3 years ago

alexander-petkov commented 3 years ago

Explore available GEOS FP data products and add them to Geoserver

Data of interest:

Variable Name Description File Time resolition Units of measure
Temperature TLML surface air temperature inst1_2d_lfo_Nx 1 hr K
Relative humidity RH relative humidity after moist inst3_3d_aer_Nv 3 hr 1
Windspeed SPEEDLML surface_wind_speed inst1_2d_lfo_Nx 1 hr m s-1
Wind Dir (to derive) U/V east/northward wind inst3_3d_asm_Nv 3 hr m s-1
Rainfall PRECTOT total precipitation tavg1_2d_flx_Nx 1 hr from 00:30 UTC kg m-2 s-1
Solar Radiation SWGDN surface incoming shortwave flux tavg1_2d_rad_Nx 1 hr from 00:30 UTC W m-2

Documentation: https://gmao.gsfc.nasa.gov/pubs/docs/Lucchesi1203.pdf

http://wiki.seas.harvard.edu/geos-chem/index.php/GEOS-FP

Data availability:

  1. Via URL: https://portal.nccs.nasa.gov/datashare/gmao/geos-fp/forecast

  2. Via OpenDAP: https://opendap.nccs.nasa.gov/dods/GEOS-5/fp/0.25_deg

alexander-petkov commented 3 years ago

Relative Humidity

There are a few RH data sets, each at diferent atm level. I picked level 72, which corresponds to 985 hPa, the average value of surface pressure on Earth.

Units:

I had the same questions as in this discussion Screenshot from 2020-10-08 13-50-09

From what I understand, multiply by 100 to get percent is needed.

Available Rel Humidity datasets:

Variable Dataset Dataset name Time resolution Begins from # of DImensions Vert level
RH inst3_3d_aer_Nv 3d instantaneous aerosol diagnostics 3-hourly 00:00 UTC 4 71 (985 hPa)
RH inst3_3d_asm_Np 3d assimilated state on pressure levels 3-hourly 00:00 UTC 4 1 (1000 hPa)

Computing RH

Compute RH using Specific humidity (QLML), Surface pressure (PS) . and Temperature (TLML) from nst1_2d_lfo_Nx (2d time-averaged land surface forcing) dataset:

  1. Get input data:
input="https://opendap.nccs.nasa.gov/dods/GEOS-5/fp/0.25_deg/fcast/inst1_2d_lfo_Nx/inst1_2d_lfo_Nx.20201021_00?... \
   tlml[0][y][x],ps[0][y][x],qlml[0][y][x]"

gdal_translate -of GTiff ${input} input.tiff

The result is a 3 band file for a single time step: Screenshot from 2020-10-23 17-06-32

  1. Calculate RH using the formula from the qair2rh function
gdal_calc.py --format=GTiff -A input.tiff --A_band=1 \
   -B input.tiff --B_band=2 \ 
   -C input.tiff --C_band=3 \
   --calc='( C * B / (0.378 * C + 0.622))/(6.112 * exp((17.67 * (A-273.15))/(A-29.65)))' \
   --outfile=rh.tif

Result: Screenshot from 2020-10-23 17-14-17

Sources:

  1. How do I convert specific humidity to relative humidity?

  2. metutils.R source code

alexander-petkov commented 3 years ago

Wind direction

This will have to be derived from UV components

Use tavg1_2d_slv_Nx: 2d time-averaged single level diagnostics U10M: 10-meter eastward wind V10M: 10-meter northward wind

Time stamps are 1-hourly from 00:30 UTC.

Maybe I can time stamp it at every hour, since the data is time-averaged.

wmjolly commented 3 years ago

Is there a 2m dewpoint temperature grid? We can calc RH from it

On Thu, Oct 8, 2020 at 1:51 PM alexander-petkov notifications@github.com wrote:

Relative Humidity:

There are a few RH data sets, each at diferent atm level. I picked level 72, which corresponds to 985 hPa, the average value of surface pressure on Earth.

Units:

I had the same questions as in this discussion https://reanalyses.org/atmosphere/merra-2-notes-questions-and-discussion [image: Screenshot from 2020-10-08 13-50-09] https://user-images.githubusercontent.com/39599557/95506526-3c23da80-096d-11eb-96d4-b4a7c79e097e.png

From what I understand, multiply by 100 to get percent is needed.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/alexander-petkov/wfas/issues/33#issuecomment-705787725, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA4G3D7QSMHYHTRM3X5IZWLSJYJ5TANCNFSM4SJEMW7A .

alexander-petkov commented 3 years ago

Is there a 2m dewpoint temperature grid? We can calc RH from it

I am not seeing dewpoint anywhere in the documentation.

alexander-petkov commented 3 years ago

Temperature

Conversion command example:


gdal_translate  \
   -of GTiff -ot Int16 -co TILED=YES -co COMPRESS=DEFLATE \
   NETCDF:"GEOS.fp.fcst.inst1_2d_lfo_Nx.20201008_06+20201008_0400.V01.nc4":TLML \
   tlml.tiff

Available Temperature datasets:

Variable Dataset Dataset name Time resolution Begins from # of DImensions Vert level Units
TLML inst1_2d_lfo_Nx 2d time-averaged land surface forcing 1-hourly 00:00 UTC 3 single level K
T2M inst3_2d_asm_Nx 2d assimilated state 3-hourly 00:00 UTC 3 single level K
T inst3_3d_asm_Np 3d assimilated state on pressure levels 3-hourly 00:00 UTC 4 1 (1000 hPa) K
TLML tavg1_2d_flx_Nx 2d time-averaged surface flux diagnostics 1-hourly 00:30 UTC 3 single level K
T2M tavg1_2d_slv_Nx 2d time-averaged single level diagnostics 1-hourly 00:30 UTC 3 single level K
T tavg3_3d_asm_Nv 3d time-averaged assimilated state on native levels 3-hourly 01:30 UTC 4 71 (985 hPa) K

At this time I am leaning towards using TLML from the first row for hourly data, or T2M from the second row for 3-hour data.

alexander-petkov commented 3 years ago

Explore archiving MERRA-2 data: https://gmao.gsfc.nasa.gov/reanalysis/MERRA-2

Maybe store this historical archive offsite

alexander-petkov commented 3 years ago

Vertical Levels

References from the documentation on p. 59:

Screenshot from 2020-10-23 13-13-29 Screenshot from 2020-10-23 13-13-48

alexander-petkov commented 3 years ago

Summary of datasets to be used

Variable Dataset Dataset name Time resolution Begins from # of DImensions Units
T2M tavg1_2d_slv_Nx 2d time-averaged single level diagnostics 1-hourly 00:30 UTC 3 K
RH inst1_2d_lfo_Nx 2d time-averaged land surface forcing 1-hourly 00:00 UTC 3 % (computed)
U10M tavg1_2d_slv_Nx 2d time-averaged single level diagnostics 1-hourly 00:30 UTC 3 m s-1
V10M tavg1_2d_slv_Nx 2d time-averaged single level diagnostics 1-hourly 00:30 UTC 3 m s-1
PRECTOT tavg1_2d_flx_Nx 2d time-averaged surface flux diagnostics 1-hourly 00:30 UTC 3 kg m-2 s-1
SWGDN tavg1_2d_rad_Nx 2d time-averaged radiation diagnostics 1-hourly 00:30 UTC 3 W m-2
CLDTOT tavg1_2d_rad_Nx 2d time-averaged radiation diagnostics 1-hourly 00:30 UTC 3 %
wmjolly commented 3 years ago

Looks like you found them?

On Mon, Oct 26, 2020 at 10:11 AM alexander-petkov notifications@github.com wrote:

Summary of datasets to be used Variable Dataset Dataset name Time resolution Begins from # of DImensions Units T2M tavg1_2d_slv_Nx 2d time-averaged single level diagnostics 1-hourly 00:30 UTC 3 K RH inst1_2d_lfo_Nx 2d time-averaged land surface forcing 1-hourly 00:00 UTC 3 % (computed) U10M tavg1_2d_slv_Nx 2d time-averaged single level diagnostics 1-hourly 00:30 UTC 3 m s-1 V10M tavg1_2d_slv_Nx 2d time-averaged single level diagnostics 1-hourly 00:30 UTC 3 m s-1 PRECTOT tavg1_2d_flx_Nx 2d time-averaged surface flux diagnostics 1-hourly 00:30 UTC 3 kg m-2 s-1 SWGDN tavg1_2d_rad_Nx 2d time-averaged radiation diagnostics 1-hourly 00:30 UTC 3 W m-2 CLDTOT tavg1_2d_rad_Nx 2d time-averaged radiation diagnostics 1-hourly 00:30 UTC 3 %

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/alexander-petkov/wfas/issues/33#issuecomment-716653562, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA4G3DYRMXALABY4JNWLWW3SMWNSRANCNFSM4SJEMW7A .

alexander-petkov commented 3 years ago

Looks like you found them?

I think I have found what I need.

A few questions remain:

  1. Should I use the hourly "surface air temperature", or Temperature stated at 2m is more appropriate?
  2. A lot of the datasets are "time averaged" and are time stamped at the bottom of the hour . Is there a problem with adjusting times tamps to the beginning of each hour?
wmjolly commented 3 years ago

2m temperature is what we want. I'm good with adjusting times to start of the hour for time averaged values.

On Mon, Oct 26, 2020 at 10:21 AM alexander-petkov notifications@github.com wrote:

Looks like you found them?

I think I have found what I need.

A few questions remain:

  1. Should I use the hourly "surface air temperature", or Temperature stated at 2m is more appropriate?
  2. A lot of the datasets are "time averaged" and are time stamped at the bottom of the hour . Is there a problem with adjusting times tamps to the beginning of each hour?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/alexander-petkov/wfas/issues/33#issuecomment-716660877, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA4G3DYUDG5IT6HA6GSZCG3SMWOZTANCNFSM4SJEMW7A .

alexander-petkov commented 3 years ago

Configured under gmao workspace. I also added the update_gmao.sh script to the daily cron update.