Design and implement a solution for generating an 8-band Flammap raster input file

[alexander-petkov]

Design and implement a solution for creating an 8-band raster (Geotiff), containing the input data for Flammmap.

Flammap is a fire spread simulation program which is originated from the Fire Science Laboratory in Missoula, MT, and it is freely available:

https://www.firelab.org/project/flammap

The 8 input band, from the Flammap documentation: Note: also add units

1. elevation
2. slope
3. aspect
4. fuel model
5. canopy cover
6. stand height
7. crown base height
8. canopy bulk density

[wmjolly]

Important design considerations:

1. Request by center coordinate (lat/long) with a tolerance +/- kilometers to define bounding box.
2. Reproject to WFDSS projection centered on the lat/long of the request.
3. Request data from any region of LANDFIRE (CONUS, Alaska, Hawaii, etc..,) without having to request from a different service.
4. Request any version of LANDFIRE (OK to work with current version now but should have hooks for old and future verisons of the data).

This is my 'wish list' but I think they are all doable.

[alexander-petkov]

Data sources

As @wmjolly mentioned above, the input bands are to be provided by Landfire data.The Landfire mosaics provided data for the initial prototype:

https://www.landfire.gov/version_comparison.php

The relevant datasets from the table in the link above are:

1. elevation (US_DEM_2016_02192019)
2. slope (US_SLP_Deg_2016_02192019)
3. aspect (US_ASP_2016_02192019)
4. fuel model--this can be either FBFM13 (13 Anderson Fire Behavior Fuel Models), or FBFM40 (40 Scott & Burgan Fire Behavior Fuel Models). I used FBMFM40 in the initial prototype.
5. Forest Canopy Cover: US_140CC_20180620
6. Forest Canopy Height: US_140CH_20180620
7. Forest Canopy Base Height: US_140CBH_20180620
8. Forest Canopy Bulk Density: US_140CBD_20180620

[alexander-petkov]

Output projection

Output projection should be Albers, where the standard parallels are provided by the lower and upper latitude bounds.

This raises another issue--is it possible to submit a Web Coverage request with its own custom projection definition?

Note Extract particulars regarding reasoning for a custom projection from email.

[alexander-petkov]

Data extent In the case of requesting dataset containing NODATA areas, the extent should be clipped such that NODATA portions are omitted. This is because Flammap does not exclude NODATA regions, as I understand.

[alexander-petkov]

This raises another issue--is it possible to submit a Web Coverage request with its own custom projection definition?

The outputCRS parameter seems to enable us to request a coverage with a custom projection.

Reference: Downloading the data in a different projection

Mapserver docs provide an example POST-XML GetCoverage request with outputCRS parameter: https://mapserver.org/ogc/wcs_server.html

Relevant discussion with examples on gis.stackexchange: https://gis.stackexchange.com/questions/284720/geoserver-does-not-work-wcs-parameters-subsettingcrs-and-outputcrs

[wmjolly]

Well that looks promising, doesn't it?

MJ

On Wed, Jul 3, 2019 at 12:50 PM alexander-petkov notifications@github.com wrote:

This raises another issue--is it possible to submit a Web Coverage request with its own custom projection definition?

The outputCRS parameter seems to enable us to request a coverage with a custom projection.

Reference: Downloading the data in a different projection https://geoserver.geo-solutions.it/edu/en/wcs/get.html#downloading-the-data-in-a-different-projection

Mapserver docs provide an example POST-XML GetCoverage request with outputCRS parameter: https://mapserver.org/ogc/wcs_server.html

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/alexander-petkov/wfas/issues/2?email_source=notifications&email_token=AA4G3D4KS36YUIKJIPLJL3DP5TYH5A5CNFSM4HSEGZR2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZFLYWQ#issuecomment-508214362, or mute the thread https://github.com/notifications/unsubscribe-auth/AA4G3D4AGEJBKHEFVI34ZF3P5TYH5ANCNFSM4HSEGZRQ .

[alexander-petkov]

I uploaded a prototype in the form of a Web Processing Service (WPS):

A custom Albers projection is constructed from input coordinates in decimal format. The standard parallels are aproximately 60 miles apart.

A raster input can be any raster, such that CONUS (as well as other areas) can be exported.

The result is an 8-band subset Geotiff with Albers projection set with parallels approximately 30 miles from the center.:

Result with a point query displayed in QGIS:

Projection info:

========================== TODO: Further testing and Flammap runs.

[wmjolly]

What machine is running this now? Can you post the IP address. I'm assuming I could hit it from I2?

On Wed, Dec 4, 2019 at 9:45 AM alexander-petkov notifications@github.com wrote:

I uploaded a prototype in the form of a Web Processing Service https://github.com/alexander-petkov/wfas/blob/master/weatherstream_wps/src/main/java/org/geoserver/wfas/wps/LandscapeExportWPS.java (WPS):

[image: Screenshot from 2019-12-04 08-29-33] https://user-images.githubusercontent.com/39599557/70160857-67b4a500-1678-11ea-85ad-59e4c00fefab.png

A custom Albers projection is constructed from input coordinates in decimal format. The standard parallels are aproximately 60 miles apart.

A raster input can be any raster, such that CONUS as well as and other areas can be exported.

The result is an 8-band subset Geotiff with Albers projection set with parallels approximately 30 miles from the center.:

[image: Screenshot from 2019-12-04 09-36-34] https://user-images.githubusercontent.com/39599557/70161643-a39c3a00-1679-11ea-8090-30e07eeafc99.png

Result with a point query displayed in QGIS:

[image: Screenshot from 2019-12-04 09-40-02] https://user-images.githubusercontent.com/39599557/70162075-5076b700-167a-11ea-9105-8ca5c44fa6f7.png

Projection info:

[image: Screenshot from 2019-12-04 09-43-03] https://user-images.githubusercontent.com/39599557/70162249-959ae900-167a-11ea-812b-a3517ec862bd.png

========================== TODO: Further testing and Flammap runs.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/alexander-petkov/wfas/issues/2?email_source=notifications&email_token=AA4G3D57WV27DELPPXRI4WTQW7NDFA5CNFSM4HSEGZR2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEF5VGRY#issuecomment-561730375, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA4G3D55JPJMBGE7KR74PMLQW7NDFANCNFSM4HSEGZRQ .

[alexander-petkov]

What machine is running this now? Can you post the IP address. I'm assuming I could hit it from I2?

This is on my dev laptop--I can deploy on I2 cluster.

[wmjolly]

No worries. Just curious. It looks AWESOME!

On Wed, Dec 4, 2019 at 12:27 PM alexander-petkov notifications@github.com wrote:

What machine is running this now? Can you post the IP address. I'm assuming I could hit it from I2?

This is on my dev laptop--I can deploy on I2 cluster.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/alexander-petkov/wfas/issues/2?email_source=notifications&email_token=AA4G3D7T2GF4MOLTFYF6FM3QXAADBA5CNFSM4HSEGZR2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEF6GWGQ#issuecomment-561802010, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA4G3DYMXGTCGABN3LZKV3LQXAADBANCNFSM4HSEGZRQ .

[wmjolly]

So, I tried un-projecting the CONUS and AK DEM so they were both in EPSG:4326. Then I used gdalbuildvrt to build a dem.vrt.

So I first converted the ArcGRID to a GeoTIFF and then Warped the GeoTIFFs using the following:

gdalwarp -overwrite -t_srs EPSG:4326 -tr 0.1 0.1 ak_dem_2010.tif ak_dem_2010_geog.tif

I downgraded the resolution for testing.

Then I used:

gdalbuildvrt dem.vrt geog/*.tif

And was able to load a single image into QGIS that showed both CONUS and AK using the same layer.

[wmjolly]

I'm wondering if we can do a VRT of VRTs that allow us to create a single, nationwide LANDFIRE dataset?

[alexander-petkov]

My thought was that we could test input coordinates for overlap (CONUS, AK, HI), and then serve the appropriate Landfire dataset based on that result.

On 12/4/19, wmjolly notifications@github.com wrote:

I'm wondering if we can do a VRT of VRTs that allow us to create a single, nationwide LANDFIRE dataset?

-- You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub: https://github.com/alexander-petkov/wfas/issues/2#issuecomment-561889355

[wmjolly]

<VRTDataset rasterXSize="3600" rasterYSize="487">
  <SRS>GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0],UNIT["degree",0.0174532925199433],AUTHORITY["EPSG","4326"]]</SRS>
  <GeoTransform> -1.7999999469736107e+02,  1.0000000000000001e-01,  0.0000000000000000e+00,  7.1454695417252012e+01,  0.0000000000000000e+00, -1.0000000000000001e-01</GeoTransform>
  <VRTRasterBand dataType="Int16" band="1">
    <NoDataValue>-32768</NoDataValue>
    <ColorInterp>Gray</ColorInterp>
    <ComplexSource>
      <SourceFilename relativeToVRT="1">geog/ak_dem_2010_geog.tif</SourceFilename>
      <SourceBand>1</SourceBand>
      <SourceProperties RasterXSize="3600" RasterYSize="224" DataType="Int16" BlockXSize="3600" BlockYSize="1" />
      <SrcRect xOff="0" yOff="0" xSize="3600" ySize="224" />
      <DstRect xOff="0" yOff="0" xSize="3600" ySize="224" />
      <NODATA>-32768</NODATA>
    </ComplexSource>
    <ComplexSource>
      <SourceFilename relativeToVRT="1">geog/us_dem_geog.tif</SourceFilename>
      <SourceBand>1</SourceBand>
      <SourceProperties RasterXSize="627" RasterYSize="289" DataType="Int16" BlockXSize="627" BlockYSize="6" />
      <SrcRect xOff="0" yOff="0" xSize="627" ySize="289" />
      <DstRect xOff="520.122420576645" yOff="198.050207240563" xSize="627" ySize="289" />
      <NODATA>-32768</NODATA>
    </ComplexSource>
  </VRTRasterBand>
</VRTDataset>

[alexander-petkov]

Today's commit contains a new version, with additional parameters for optional downscaling (down to 10x coarser resolution), as well as specifying a custom extent in miles, between 5 and 60:

The result is an 8-band Landscape subset file, with NoData regions preserved. I was having difficulties up until now preserving NoData areas after a "Scale" coverage operation.

[wmjolly]

Is it deployed to the server? I can't seem to hit the 192.168.59.18 machine anymore.

On Wed, Dec 11, 2019 at 3:27 PM alexander-petkov notifications@github.com wrote:

Today's commit https://github.com/alexander-petkov/wfas/blob/d33e6f0e2414a8b65648b823e0588aaafb62c36a/weatherstream_wps/src/main/java/org/geoserver/wfas/wps/LandscapeExportWPS.java contains a new version, with additional parameters for optional downscaling (down to 10x coarser resolution), as well as specifying a custom extent in miles, between 5 and 60:

[image: Screenshot from 2019-12-11 15-02-21] https://user-images.githubusercontent.com/39599557/70665551-05d0de00-1c2a-11ea-91e1-7cfbfb5bb45f.png

The result is an 8-band Landscape subset file, with NoData regions preserved. I was having difficulties up until now preserving NoData areas after a "Scale" coverage operation.

[image: Screenshot from 2019-12-11 15-03-43] https://user-images.githubusercontent.com/39599557/70665765-8263bc80-1c2a-11ea-80e4-e3a47b014d47.png

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/alexander-petkov/wfas/issues/2?email_source=notifications&email_token=AA4G3D64HOK7AXK4U2AKCOTQYFSNNA5CNFSM4HSEGZR2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGUZKJA#issuecomment-564761892, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA4G3DZTH57WGBRPLMEXITDQYFSNNANCNFSM4HSEGZRQ .

[alexander-petkov]

Is it deployed to the server? I can't seem to hit the 192.168.59.18 machine anymore.

I turned VM instances off for now, while updating the Ceph hardware.

This was on my dev laptop.

[wmjolly]

Sounds good. Thanks!

On Wed, Dec 11, 2019 at 3:47 PM alexander-petkov notifications@github.com wrote:

Is it deployed to the server? I can't seem to hit the 192.168.59.18 machine anymore.

I turned VM instances off for now, while updating the Ceph hardware.

This was on my dev laptop.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/alexander-petkov/wfas/issues/2?email_source=notifications&email_token=AA4G3D6EA3BQK2AY23UBQ6TQYFUXRA5CNFSM4HSEGZR2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGU2ZUA#issuecomment-564767952, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA4G3D7RA3MWFP2H6GQ6IBDQYFUXRANCNFSM4HSEGZRQ .

[alexander-petkov]

The Geoserver cluster is running again, with the latest version of the Landscape WPS.

Screenshot with sample values:

[alexander-petkov]

1. Modify the 8-band export such that after reprojecting, there are no areas with NODATA.
2. Set NODATA to -9999 in the output.

Edit: These changes are in this commit

[alexander-petkov]

TODO:

1. Add functionality to be able to choose Landscape export for Alaska and Hawaii, in addition to CONUS.

2. Add functionality so users are able to pick different Landfire versions.

[alexander-petkov]

I implemented the items in the comment above. These are the changes:

1. The raster input is eliminated from the user interface. Now a region for the 8-band raster is picked based on input user coordinates. Overlap is tested over CONUS, AK, and HI. The shapefile boundaries for each Landfire region are included in the jar file.

2. Landfire version can be picked from a dropdown list (or specified in a XML file). Default version is 1.4.0 which is the only one I have configured so far over all 3 regions (on Plume). I will be including other versions after some testing.

[alexander-petkov]

The shapefile boundaries for each Landfire region are included in the jar file.

This is not going to work in a cluster environment--whatever instance opens the shapefile first, it puts a lock on it. Then the process fails on other geoserver instances.

[alexander-petkov]

The shapefile boundaries for each Landfire region are included in the jar file.

This is not going to work in a cluster environment--whatever instance opens the shapefile first, it puts a lock on it. Then the process fails on other geoserver instances.

Fixed in commit 2e4fa95. Deployed on Plume.

[alexander-petkov]

Important design considerations:

1. Request by center coordinate (lat/long) with a tolerance +/- kilometers to define bounding box.

2. Reproject to WFDSS projection centered on the lat/long of the request.

3. Request data from any region of LANDFIRE (CONUS, Alaska, Hawaii, etc..,) without having to request from a different service.

4. Request any version of LANDFIRE (OK to work with current version now but should have hooks for old and future verisons of the data).

This is my 'wish list' but I think they are all doable.

All requirements above are now implemented.

I need to finish downloading the data for all 3 regions and versions. Right now I have all versions for Hawaii configured on Plume:

[alexander-petkov]

Added a mimetype (image/tif) in this commit that allows for Landscape export with a .tif extension.

[alexander-petkov]

Duplicate the LandscapeExport service for Morocco We will need 'filler' layers in the meantime: a layer of all zeros for CBD, CBH, and CC

[alexander-petkov]

Canopy Cover (CC) for Morocco

Derived from CH:

gdal_calc.py --format=GTiff ${GDAL_CALC_OPTIONS} \
   --type=Byte --NoDataValue=255 \
   -A mar_ch_final2.tif \
   --calc='where((A>0),50,0)' \
   --outfile=mar_cc_final2.tif

If CH>0, assume 50 per cent, otherwise zero.

[alexander-petkov]

Canopy Base Height (CBH) for Morocco

Derived from CH:

gdal_calc.py --format=GTiff ${GDAL_CALC_OPTIONS} \
   --type=Float32 -A mar_ch_final2.tif \
   --calc='A*0.5' \
   --outfile=mar_cbh_final2.tif

[alexander-petkov]

Canopy Height (CH) for Morocco

Data source: https://glad.geog.umd.edu/Potapov/Forest_height_2019/Forest_height_2019_NAFR.tif

Clipping the file to the extent of the mar:Pais layer, and reprojecting to EPSG:26191:

export GEOTIFF_OPTIONS="-co PROFILE=GeoTIFF -co COMPRESS=DEFLATE -co TILED=YES \
   -co BLOCKXSIZE=256 -co BLOCKYSIZE=256 \
   -co NUM_THREADS=ALL_CPUS"

gdalwarp ${GEOTIFF_OPTIONS} -of GTiff -t_srs EPSG:26191  \
   -te -17.105316329584923 20.770846080881995 -1.0030470646702767 35.925029366711975 \
   -te_srs WGS84 \
   /vsicurl/https://glad.geog.umd.edu/Potapov/Forest_height_2019/Forest_height_2019_NAFR.tif \
   Forest_height_2019_NAFR_mor.tif

[alexander-petkov]

Canopy Bulk Density (CBD) for Morocco

Derived from CH:

gdal_calc.py --overwrite --format=GTiff ${GDAL_CALC_OPTIONS} \
   --type=Float32 -A mar_ch_final2.tif \
   --calc='numpy.power(A,3)*0.1' \
   --outfile=mar_cbd_final2.tif