search

ariesteam / aries

http://www.ariesonline.org
GNU General Public License v3.0
6 stars 1 forks source link

SurfaceDiversionCapacity loses info and is spatially shifted incorrectly #3

Closed lambdatronic closed 13 years ago

lambdatronic commented 13 years ago

The core.models.water-san-pedro/surface-diversions model is meant to perform a kbox query for a measurement of waterSupplyService:SurfaceDiversionCapacity in mm. The code reads as follows:

(defmodel surface-diversions SurfaceDiversionCapacity
  (measurement SurfaceDiversionCapacity "mm"))

Running this model in dump mode in the _core.contexts.beta/san_pedrous512 region, produces this output:

2011-05-18 17:07:20,773 [main] INFO  org.integratedmodelling.geospace.Geospace reading WCS source http://ecoinformatics.uvm.edu/geoserver/wcs#sanPedro:Diversions2
accessing: http://ecoinformatics.uvm.edu/geoserver/wcs?service=WCS&version=1.0.0&request=DescribeCoverage&coverage=sanPedro:Diversions2
2011-05-18 17:07:20,859 [main] INFO  org.integratedmodelling.geospace.Geospace parsing descriptor for sanPedro:Diversions2
WCS URL: http://ecoinformatics.uvm.edu/geoserver/wcs?service=WCS&version=1.0.0&request=GetCoverage&coverage=sanPedro:Diversions2&bbox=-111.012,31.328,-109.845,33.281&crs=EPSG:4326&responseCRS=EPSG:4326&width=306&height=512&format=geotiff
Spatially distributed on a 306 by 512 raster grid (cell area = 97,756.737 m²)
----------------------------------------------------------------------------
waterSupplyService:SurfaceDiversionCapacity
----------------------------------------------------------------------------
[0 0]   |                                                         |     434
[0 0]   |                                                         |       0
[0 0]   |                                                         |       0
[0 0]   |                                                         |       0
[0 0]   |                                                         |       0
[0 0]   |                                                         |       0
[0 0]   |                                                         |       0
[0 0]   |                                                         |       0
[0 0]   |                                                         |       0
[0 0]   |                                                         |       0
----------------------------------------------------------------------------
no-data |*********************************************************|  156238
Aggregated over 15,265.692 km²: 0 mm

The 434 non-no-data pixels are all 0s. This was confirmed by adding a simple binary-coding model wrapper around the surface-diversions model, defined as follows:

(defmodel sdwrapper SurfaceDiversionCapacityCode
  (binary-coding SurfaceDiversionCapacityCode
                 :context (surface-diversions)
                 :state   #(cond (no-data? (:surface-diversion-capacity %)) 0
                                 (zero? (:surface-diversion-capacity %)) 0
                                 :otherwise 1)))

Running this model in dump mode in the same context, produces this histogram:

----------------------------------------------------------------------------
waterSupplyService:SurfaceDiversionCapacityCode
----------------------------------------------------------------------------
[0 0]   |*********************************************************|  156672
[0 0]   |                                                         |       0
[0 0]   |                                                         |       0
[0 0]   |                                                         |       0
[0 0]   |                                                         |       0
[0 0]   |                                                         |       0
[0 0]   |                                                         |       0
[0 0]   |                                                         |       0
[0 0]   |                                                         |       0
[0 0]   |                                                         |       0
----------------------------------------------------------------------------
no-data |                                                         |       0

As all no-data values were set to 0, this indicates that the other 434 were also 0s to begin with.

However, this zeroing of values is not coming from Geoserver. According to the INFO output generated by these model commands, ARIES is sending a request to our Geoserver with this URL:

http://ecoinformatics.uvm.edu/geoserver/wcs?service=WCS&version=1.0.0&request=GetCoverage&coverage=sanPedro:Diversions2&bbox=-111.012,31.328,-109.845,33.281&crs=EPSG:4326&responseCRS=EPSG:4326&width=306&height=512&format=geotiff

Pasting this URL into my web browser (Chrome 10.0.648.127) yields an immediate download of the Diversions2.tiff file. Opening this file in GRASS, I see the correct values as displayed by Geoserver's Layer Preview feature at this URL:

http://ecoinformatics.uvm.edu/geoserver/wms?service=WMS&version=1.1.0&request=GetMap&layers=sanPedro:Diversions2&styles=&bbox=-110.255,31.803,-110.205,31.923&width=213&height=512&srs=EPSG:4326&format=application/openlayers

Specifically, two non-zero values are defined: one in the upper left corner of the layer and one in the lower right corner. However, if I rerun core.models.water-san-pedro/surface-diversions using the NetCDF output mode, and then load this layer into GRASS, I see a layer which contains only 0s (434 to be exact) and nils. Furthermore, the output layer's bounding box (although specified correctly in the kbox XML and printed correctly by the observe command at the ARIES shell) does not match that of the layer from Geoserver. It is shifted quite a bit northward of the correct location as can be seen in this raster overlay (click it to see the original larger version of the image).

This needs to be corrected in order to run the San Pedro surface water models (and potentially any other models with a similar use type).

fvilla commented 13 years ago

Looking at this, the code is all correct, but apparently the TIFFs returned by the query have a different bounding box than requested. The one from the query above has its origin at -111.01009, 33.2790 - while thinklab uses the original context throughout. This is plenty to shift the few pixels >0 outside of the coverage of the original bounding box, thereby turning then to NaNs.

Now I really don't know how to fix that - there's nothing to be fixed at the thinklab side, and the way this works makes really impossible to change the context in midstream. I need to understand what adjustments gs makes and whether they're systematic and predictable, but what I expect is that it "fixes" coordinates based on a full-pixel rounding, and if that's the case, it's going to be different for each layer therefore unfixable at the source (adjusting the context before model computation).

OR there may be a way to get GS to apply the proper transformations instead of adjusting the context. I expect this may be a strategy to reduce server load on GS side and there may be options to switch that off. Need to investigate - help would be appreciated.

fvilla commented 13 years ago

According to this thread:

http://osgeo-org.1803224.n2.nabble.com/GeoServer-s-accuracy-when-using-WCS-and-Geotiffs-td6133783.html

the problem is known and things are significantly better in GS 2.1.x (the first official release was announced last week or so). Apparently switching to 2.1 could help fix this one. Should we try?

lambdatronic commented 13 years ago

Upgrading Geoserver is certainly a possibility. The geo-xls script makes that much less of a pain than before, but there are still likely to be a few hiccups. I need to reorganize some Geoserver on huginn anyway, so perhaps I can do all of this at the same time.

Just to be sure though, you did understand that the image I'm displaying in red above shows two TIFFs:

  1. The one returned by Geoserver from this URL: http://ecoinformatics.uvm.edu/geoserver/wcs?service=WCS&version=1.0.0&request=GetCoverage&coverage=sanPedro:Diversions2&bbox=-111.012,31.328,-109.845,33.281&crs=EPSG:4326&responseCRS=EPSG:4326&width=306&height=512&format=geotiff
  2. The one returned by ARIES after performing contextualization.

These are different, but the Geoserver one looks correct in GRASS. That says to me that the problem is happening during contextualization. I'm not sure I understand quite what you are describing in terms of the shifted bounding box here. Perhaps you could point me at the proper metadata to check for this?