WikiWatershed / mmw-geoprocessing

A Spark Job Server job for Model My Watershed geoprocessing.
Apache License 2.0
6 stars 6 forks source link

Fallback to empty tile for queries out of bound #90

Closed rajadain closed 4 years ago

rajadain commented 4 years ago

Overview

In cases when one layer has a larger extent than another, such as NLCD and ARA, it is possible to get Spatial Keys from one which are not available in the other. This would previously cause a crash. Now we default to an empty tile (with NODATA values) instead.

Also add an example request of a square shape in Phoenix, AZ which is within NLCD but outside ARA. This request would previously cause an Index Out of Bounds error. Now we get back correct values, with NODATA for ARA.

Connects https://github.com/WikiWatershed/model-my-watershed/issues/3239

Demo

On develop:

$ http :8090/run < scratch/EmptyTileRequest.json

HTTP/1.1 500 Internal Server Error
Content-Length: 67
Content-Type: text/plain; charset=UTF-8
Date: Tue, 28 Jan 2020 16:51:04 GMT
Server: akka-http/10.0.9

java.util.NoSuchElementException: key not found: SpatialKey(66,132)

image

On this branch:

$ http :8090/run < examples/EmptyTileRequest.json 

HTTP/1.1 200 OK
Content-Length: 121
Content-Type: application/json
Date: Tue, 28 Jan 2020 16:50:17 GMT
Server: akka-http/10.0.9

{
    "result": {
        "List(21, -2147483648)": 8,
        "List(22, -2147483648)": 82,
        "List(23, -2147483648)": 462,
        "List(24, -2147483648)": 556
    }
}

image

Notes

MMW will have to be updated to handle the NODATA values, which currently treats them as 0 instead of "No Data". For example, on develop: This has been done in https://github.com/WikiWatershed/model-my-watershed/pull/3241.

image

On this branch:

image

We use int NODATA, not the float one, because this will only apply to the keys of the result. In the result, the values can be int or float, but the keys are always ints.

Testing Instructions

Though not necessary, in case you'd like to try this with MMW, the way to test it is:

$ ./scripts/cibuild.sh
$ mv api/target/scala-2.11/api-assembly-4.0.1.jar $MMW_ROOT/mmw-geoprocessing-4.0.1.jar
$ cd $MMW_ROOT
$ vagrant ssh worker -c 'sudo mv /vagrant/*.jar /opt/geoprocessing/'
$ vagrant ssh worker -c 'sudo service mmw-geoprocessing stop'
$ vagrant ssh worker -c 'sudo service mmw-geoprocessing start'

And then go to :8000/, which will use the new geoprocessing service.

rajadain commented 4 years ago

@lossyrob Could you take a quick look at this please?

rajadain commented 4 years ago

Thanks for taking a look!