alexander-petkov / wfas

A placeholder for the WFAS project.
4 stars 1 forks source link

Add ingest for the Quantitative Precipitation Forecast #32

Open wmjolly opened 3 years ago

wmjolly commented 3 years ago

https://www.wpc.ncep.noaa.gov/qpf

GRIB2 files are here

alexander-petkov commented 3 years ago

WPC Quantitative Precipitation Forecast on noaa.gov: https://www.wpc.ncep.noaa.gov/qpf/qpf2.shtml

Grib2 ftp location: https://ftp.wpc.ncep.noaa.gov/2p5km_qpf/

alexander-petkov commented 3 years ago

Spatial info and pixel resolution appears identical to NDFD, slightly larger grid

alexander-petkov commented 3 years ago

Consider this table when scheduling a cron update:

Screenshot 2020-10-02 at 8 45 10 AM

alexander-petkov commented 3 years ago

Naming convention: https://www.wpc.ncep.noaa.gov/grib/hpcgrib022301.html

Screenshot 2020-10-02 at 9 03 03 AM

alexander-petkov commented 3 years ago

For now I will get the 6-hour data for the 00Z forecast--this will result in 7-day forecast with time resolution for every 6 hours.

alexander-petkov commented 3 years ago

Get date in YYYYMMDD format:

date +'%Y%m%d'

Get a list of today's files for the 00 forecast:

today=`date +'%Y%m%d' `
curl -s --list-only  https://ftp.wpc.ncep.noaa.gov/2p5km_qpf/|grep -E "p06m_${today}00f[
0-9]{3}.grb"|cut -d '"' -f 2

Or just use a for loop without querying remote server for a list:

for h in `seq -w 006 6 168`
alexander-petkov commented 3 years ago

It would be nice to snag a ready colormap

alexander-petkov commented 3 years ago

So far I have added the 6-hour accumulation files, for 7 days.

Are any others needed?

wmjolly commented 3 years ago

Are there 24 hour summaries too?

On Fri, Oct 2, 2020, 14:37 alexander-petkov notifications@github.com wrote:

So far I have added the 6-hour accumulation files, for 7 days.

Are any others needed?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/alexander-petkov/wfas/issues/32#issuecomment-702969100, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA4G3D6HUU5B6U72RURVU4TSIZBYHANCNFSM4Q4SDZ2Q .

alexander-petkov commented 3 years ago

Are there 24 hour summaries too?

Yes

alexander-petkov commented 3 years ago

Is the 24 hour forecast the one that is needed?

Should I drop the 6-hour files, or keep it?

wmjolly commented 3 years ago

Keep all the 6 hour files too. They'll be useful in the future. However, the 24 hour summaries are most useful right now.

On Mon, Oct 5, 2020 at 9:48 AM alexander-petkov notifications@github.com wrote:

Is the 24 hour forecast the one that is needed?

Should I drop the 6-hour files, or keep it?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/alexander-petkov/wfas/issues/32#issuecomment-703719009, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA4G3D55DI5GPGGP2PDU7GLSJHTEDANCNFSM4Q4SDZ2Q .

alexander-petkov commented 3 years ago

I copied the colormap from the WPC images by using a colorpicker tool.

Tedious, but it works.... The values in the sld file need to be in ascending order, if modes like ramp or interval are used:

<ColorMap type="ramp">
              <ColorMapEntry color="#ffffff" quantity="0.00" label="0.00" opacity="0" />
              <ColorMapEntry color="#7fff00" quantity="0.01" label="0.01" />
              <ColorMapEntry color="#00cd00" quantity="0.10" label="0.10" />
              <ColorMapEntry color="#008b00" quantity="0.25" label="0.25" />
              <ColorMapEntry color="#104e8b" quantity="0.50" label="0.50" />
              <ColorMapEntry color="#1e90ff" quantity="0.75" label="0.75" />
              <ColorMapEntry color="#00b2ee" quantity="1.00" label="1.00" />
              <ColorMapEntry color="#00eeee" quantity="1.25" label="1.25" />
              <ColorMapEntry color="#8968cd" quantity="1.50" label="1.50" />
              <ColorMapEntry color="#912cee" quantity="1.75" label="1.75" />
              <ColorMapEntry color="#8b008b" quantity="2" label="2.00" />
              <ColorMapEntry color="#8b0000" quantity="2.50" label="2.50" />
              <ColorMapEntry color="#cd0000" quantity="3" label="3.00" />
              <ColorMapEntry color="#ee4000" quantity="4" label="4.00" />
              <ColorMapEntry color="#ff7f00" quantity="5" label="5.00" />
              <ColorMapEntry color="#cd8500" quantity="7" label="7.00" />
              <ColorMapEntry color="#ffd700" quantity="10" label="10.00" />
              <ColorMapEntry color="#ffff00" quantity="15" label="15.00"/>
              <ColorMapEntry color="#ffaeb9" quantity="20" label="20.00" />

However, my maps look quite different than WPC.

Mine:

image

WPC:

image

I think their maps aren't using all these colors. despite what the legend says.

alexander-petkov commented 3 years ago

Style colors scraped from QPF KML files:

Value Color
0.1 #7fff00
0.25 #00cd00
0.5 #008b00
0.75 #104e8b
1.0 #1e90ff
1.25 #00b2ee
1.5 #00eeee
1.75 #8968cd
2.0 #912cee
2.5 #8b008b
3.0 #8b0000
4.0 #cd0000
5.0 #ee4000
alexander-petkov commented 3 years ago

Fixed url encoding error for coverage names with spaces.

wmjolly commented 3 years ago

[image: image.png] So cool!

On Wed, Oct 7, 2020 at 4:08 PM alexander-petkov notifications@github.com wrote:

Fixed url encoding error for coverage names with spaces.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/alexander-petkov/wfas/issues/32#issuecomment-705219590, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA4G3D4XPPYPSHEBXXMYDNDSJTRGPANCNFSM4Q4SDZ2Q .