AtlasOfLivingAustralia / layers-service

Spatial layers - this repo for issues/doc only, not code
3 stars 4 forks source link

Load Layer - Indigenous forest estate #70

Closed rp-ala closed 7 years ago

rp-ala commented 8 years ago

For the first time, a comprehensive national description of the forest areas over which Indigenous people have ownership, management or special rights is available. This new spatial dataset describes and maps Australia's Indigenous forest estate. The Indigenous forest dataset describes the 41 million hectares of forested land that is owned or managed by Australia's indigenous communities or over which Indigenous people have use or rights. The dataset was developed by combining data on Indigenous land with data on Australia’s forest cover. Approximately three-quarters of this 41 million hectares of Indigenous forest is located in the Northern Territory and Queensland.

http://data.daff.gov.au/anrdl/metadata_files/pb_aif13d9abfs20150828_11a.xml

ansell commented 8 years ago

Another document referenced in the metadata is:

http://data.daff.gov.au/data/warehouse/9aaf/aif/2013/aif13d9abfs20150828/IndigenousForestEstate.docx

ansell commented 8 years ago

Eventually found another location that has a shapefile download on it using some google searches for ABARES and browsing to get to it:

http://www.agriculture.gov.au/abares/forestsaustralia/publications/display?url=http://143.188.17.20/anrdl/DAFFService/display.php?fid=pb_aif13g9abfs20140604_11a.xml

ansell commented 8 years ago

Looking at the URLs though, it is possible that page doesn't include the same version as the XML document, but the equivalent page for the XML document (dated 2015-08-28) in the original issue has no shapefiles attached to it, and merely references the 2014 version of the metadata, and both are labelled as 2013 internally.

http://www.agriculture.gov.au/abares/forestsaustralia/publications/display?url=http://143.188.17.20/anrdl/DAFFService/display.php?fid=pb_aif13d9abfs20150828_11a.xml

ansell commented 8 years ago

Provided using ArcGIS format (.adf) files.

Format description is at:

http://www.digitalpreservation.gov/formats/fdd/fdd000281.shtml

ansell commented 8 years ago

Failure in spatial-service when uploading this layer to spatial-test.ala.org.au. Opened issue on spatial-service at https://github.com/AtlasOfLivingAustralia/spatial-service/issues/7

ansell commented 8 years ago

Looking into conversion possibilities using gdal to convert the .adf files to ESRI hdr files or others that are supported by spatial.ala.org.au if that doesn't work:

http://www.gdal.org/formats_list.html

ansell commented 8 years ago

Seemed to successfully create an ENVI HDR .bil file after extracting the zip file and running gdal_translate on the ind_for13 directory

$ ls IndigenousForestEstate/aif13g9abfs20131231egialb132/ind_for13
dblbnd.adf  metadata.xml  sta.adf  w001000.adf   w001001.adf   z001001.adf   z001002.adf   z001003.adf
hdr.adf     prj.adf       vat.adf  w001000x.adf  w001001x.adf  z001001x.adf  z001002x.adf  z001003x.adf

$ gdal_translate -of ENVI IndigenousForestEstate/aif13g9abfs20131231egialb132/ind_for13/ target/test.envi.hdr.bil
Input file size is 41478, 39174
0...10...20...30...40...50...60...70...80...90...100 - done.

$ gdalinfo target/test.envi.hdr.bil
Driver: ENVI/ENVI .hdr Labelled
Files: target/test.envi.hdr.bil
       target/test.envi.hdr.bil.aux.xml
       target/test.envi.hdr.hdr
Size is 41478, 39174
Coordinate System is:
PROJCS["Albers",
    GEOGCS["GCS_GDA_1994",
        DATUM["Geocentric_Datum_of_Australia_1994",
            SPHEROID["GRS_1980",6378137,298.257222101]],
        PRIMEM["Greenwich",0],
        UNIT["Degree",0.017453292519943295]],
    PROJECTION["Albers_Conic_Equal_Area"],
    PARAMETER["standard_parallel_1",-18],
    PARAMETER["standard_parallel_2",-36],
    PARAMETER["latitude_of_center",0],
    PARAMETER["longitude_of_center",132],
    PARAMETER["false_easting",0],
    PARAMETER["false_northing",0],
    UNIT["METERS",1]]
Origin = (-1938200.000000000000000,-969800.000000000000000)
Pixel Size = (100.000000000000000,-100.000000000000000)
Metadata:
  Band_1=Band 1
Image Structure Metadata:
  INTERLEAVE=BAND
Corner Coordinates:
Upper Left  (-1938200.000, -969800.000) (115d 0'11.57"E,  8d 3'42.47"S)
Lower Left  (-1938200.000,-4887200.000) (108d49'12.91"E, 43d 6'15.58"S)
Upper Right ( 2209600.000, -969800.000) (151d20'33.95"E,  7d42' 1.98"S)
Lower Right ( 2209600.000,-4887200.000) (158d20'22.97"E, 42d37' 8.56"S)
Center      (  135700.000,-2928500.000) (133d23' 3.27"E, 27d 0'30.29"S)
Band 1 Block=41478x1 Type=Int16, ColorInterp=Undefined
  NoData Value=-32768
ansell commented 8 years ago

Spatial-service is not currently able to support this. After looking through the spatial-service code, as there is no documentation, I have tried both ENVI HDR and ESRI HDR formats that the internal code seems to support but both of them fail with the exception described in AtlasOfLivingAustralia/spatial-service#7

ansell commented 8 years ago

I wasn't packaging the .prj file along with the .bil and .hdr files, which was causing an issue.

Uploaded to spatial-test at:

http://spatial-test.ala.org.au/spatial-service/manageLayers/layer/10842

http://spatial-test.ala.org.au/spatial-service/manageLayers/field/el10842

ansell commented 8 years ago

The commands used for converting were:

$ unzip aif13g9abfs20131231egialb132.zip
$ gdal_translate -of EHdr aif13g9abfs20131231egialb132/ind_for13/ indigenous_forest_estate.bil
$ zip -r indigenous_forest_estate_processed.EHdr.zip indigenous_forest_estate.*
ansell commented 8 years ago

This layer requires attention from @adam-collins as I am unfamiliar with the use of a raster file as a contextual layer. See notes on AtlasOfLivingAustralia/spatial-service#7 for background.

ansell commented 7 years ago

Note that I deleted the layer on spatial-test as it was uploaded as an environmental layer instead of a contextual layer.

ansell commented 7 years ago

The target for this is now the 2013 v2.0 file:

http://www.agriculture.gov.au/abares/forestsaustralia/publications/display?url=http://143.188.17.20/anrdl/DAFFService/display.php?fid=pb_aif13g9abfs20160527_11a.xml

As found on:

http://www.agriculture.gov.au/abares/forestsaustralia/forest-data-maps-and-tools/forest-data

ansell commented 7 years ago

This has been loaded to spatial-test at:

http://spatial-test.ala.org.au/spatial-service/manageLayers/field/cl10900

ansell commented 7 years ago

Refs https://github.com/AtlasOfLivingAustralia/seasonal-calendar/issues/39