Enable specification of cell area rather than number of cells on the side of a context

[kbagstad]

@fvilla, @lambdatronic

I'm running into problems where we need to run a high-resolution analysis on a larger spatial context than the flow models are capable of running on (i.e., Puget Sound). One way to do this is to run multiple "tiles" (i.e., individual contexts) at a particular resolution and add the results together. Problem is, I'm too often left guessing at the right resolution to use to get a particular cell size. It would be FAR easier if rather than specifying the number of output cells on the side of a spatial context, we could specify the number of meters on the side of a square cell in the outputs, i.e., enter "30" for 30x30 m analysis, "200" for 200x200 m analysis). We know the cell size is stored during a model run, so figure this ought to be possible.

Ferd, can you let us know how complex a problem this is? It would be a big improvement in usability for modelers.

[fvilla]

Done - you can now specify a string instead of a number, e.g. "4 km" in a (grid ...) form. The string needs to contain a number, a space and a unit that can resolve to meters - so m, km, ft etc are all fine (no degrees though). Those are taken as the LINEAR size of the grid cell, not the area.

If you specify a number instead, things will work like they used to. So

(grid 128 "EPSG....") -> 128 cells on major axis (grid "1 km" "....") -> 1 square km cell (1 km * 1 km) (grid "300 m" "....") -> 90000 square meter cell

The system will translate the unit to meters, compute that as the cell width and height and compute a grid accordingly. I need to look into the reported cell h/w but that's another issue, so I'm closing this one.

[kbagstad]

OK, reopening closed issues is a bad habit to get into, but there's something really weird afoot with the new model results I've been testing today. Essentially all the results I've generated today are shifted in some strange way they shouldn't be, and the only thing that I can think of that could be responsible is the changes to how contexts are defined. If you have any old results lying around, try re-running them now and comparing. Or try running them against another known data layer. I could give specific examples, but they really don't seem necessary, literally everything has a shift on that's making the results quite screwy looking.

Thanks, Ken

[fvilla]

It has nothing to do with how contexts are defined - this is just another way to specify a resolution and it cannot change the results. What has changed (and I abundantly warned that ALL results were going to look very different) is that now I put in lots of messy code to ensure that the cells are "square" when measured in meters and not when measured in whatever original projection the context came in (decimal lat/lon in all our contexts). Because what you see as final results comes in lat/lon as the "unified" projection, making the cells square in meters will produce results that are not square in lat/lon. Of course nothing is really square because the earth isn't flat, and the implications of this seem to be unclear to everyone.

Now the point is not what is right but what you want and what the software you use expects. If you see things that are WRONG in Arc, it means that Arc makes assumptions on cells and of course going through two reprojections that ensure that they're square in meters makes them non-square in the final projection. Which may or may not make them wrong according to what it can handle. My (few) tests with my own image generators looked OK (superficially at least). And it's unclear to me that the seeing similar x and y cell size in meters means that the flow models will be "right". It's a much deeper issue, no projection is right, they all flatten an ellipsoid differently and I really don't know what should be done.

So we need to decide what we want to do. I can easily go back to the previous way and/or condition the behavior to a configuration parameter or option. This requires an assessment of whether (1) the results are wrong as opposed to different, which most likely depends on what Arc can handle, and (2) whether seeing "square" cells in meters is really crucial to the flow models working properly.

Let me know. I'm adding Brian and Gary and will be putting in the previous behavior as an option while we decide.

@lambdatronic @bvoigt

[lambdatronic]

The flow models are based on grid calculations, which all assume that the underlying cells are square. Furthermore, they all use the cell-width and cell-height values in meters to drive their calculations. Therefore, I strongly recommend that square cells in meters be the standard behavior. Projections are simply different ways of viewing 3D surfaces on flat media, but these models are moving matter and energy around between cells on a scale that is generally sufficiently small (e.g. 1km on a side) that the roundness of the Earth is largely a non-issue. However, if the metric measurements of the cells are non-square (as they have been), we are going to get really, really weird results with a lot of the models. Views and coastal storm protection are going to be particularly prone to this problem.

[kbagstad]

Well do all our contexts have to be lat/long? I tried reprojecting a test context to WGS 84 UTM Zone 10N (meters), generating WKT, and running the command:

model -o test14.nc core.models.sediment-puget/vegetation-type core.contexts.puget/wria-skokomish-test

get the error:

resource not found: cannot observe any of the 1 contingencies of model core.models.sediment-puget/vegetation-type

I take it as it stands now we have to run contexts in lat/long? Would the problem be more tractable if we could run models on contexts using spatial reference systems based in meters (assuming that's even possible)?

[kbagstad]

FWIW I also tried reprojecting after-the-fact results into a meter-based projection (UTM Zone 10N WGS 84 for Puget Sound results) and those don't remotely line up either. Just in case anyone else was thinking of trying that.

[fvilla]

See comments in email for whole story. For the moment, put back previous behavior (square cells in context's projection) with correction by Gary (i.e., cells a little more square than before). Squaring cells in meters still possible but not enabled by default: set square.cells.meters = true in the geospace plugins' properties to select it - which will work fine with everything, but will create outputs that don't work with Arc.

Not closing the issue because it is an unclosable one unless the earth becomes a sphere, Arc becomes good, or flow models become capable of handling arbitrary cells. Plus I'm pretty sure that Ken will reopen it as soon as any other problem, related or not, pops up.

[kbagstad]

Well seems closeable given that the Earth's shape is unlikely to change, Arc is unlikely to improve, and changing the flow models to handle arbitrary cells will be an issue for Gary this summer. Plus I don't want to be accused of irrationally keeping every issue open.