wri / gfw-datapump

Nightly batch process to generate summary statistics for user AOIs
3 stars 2 forks source link

Link results from geotrellis nightly runs with User Dashboards #2

Closed thomas-maschler closed 4 years ago

thomas-maschler commented 4 years ago

Is your feature request related to a problem? Please describe. Describe the solution you'd like Link results from geotrellis nightly runs with User Dashboards

Additional context Gluing things together. Step function: fetch geometries from the API, include in the directory that has user submitted geometries, include in the existing analysis, monitor that job succeeded, inject into the api and notify dashboard service that run was successful

thomas-maschler commented 4 years ago

Here the Geotrellis options:

Option Type Analysis or Feature Type Description
analysis string Type of analysis to run [annualupdate, annualupdate_minimal,carbonflux,gladalerts,treecoverloss]
features string all (required) URI of features in TSV format
output string all (required) URI of output dir for CSV files
feature_type string all (required) Feature type: one of 'gadm', 'wdpa', 'geostore' or 'feature
limit int all Limit number of records processed
iso_first string for gadm or wdpa feature types Filter by first letter of ISO code
iso_start string for gadm or wdpa feature types Filter by ISO code larger than or equal to given value
iso_end string for gadm or wdpa feature types Filter by ISO code smaller than given value
iso string for gadm or wdpa feature types Filter by country ISO code
admin1 string for gadm feature types Filter by country Admin1 code
admin2 string for gadm feature types Filter by country Admin2 code
id_start int for feature feature types Filter by IDs larger than or equal to given value
id_end int for feature feature types Filter by IDs smaller than given value
iucn_cat string for wdpa feature types Filter by IUCN Category
wdpa_status string for wdpa feature types Filter by WDPA Status"
tcd int for treecover analysis Select tree cover density year
threshold int for treecover analysis Treecover threshold to apply
primary-forests flag for treecover analysis Include Primary Forests
tcl flag all, requires boolean tcl field in input feature class Filter input feature by TCL tile extent
glad flag all, requires boolean glad field in input feature class GLAD tile extent
change_only flag all except `treecover loss Process change only
build_data_cube flag glad Build XYZ data cube

You will want to run the annualupdate_minimal and gladalerts analysis for now.

your commands would look like this:

org.globalforestwatch.summarystats.SummaryMain --features s3://path/to/files/*.tsv --output s3://gfw-pipelines/geotrellis/results/geostore/<runid> --feature_type geostore --analysis annualupdate_minimal

logoul commented 4 years ago

https://github.com/wri/gfw-user-aoi-batch/pull/1