Differencing two curvilinear datasets (easier than it seems!)

[karlmsmith]

Reported by steven.c.hankin on 7 May 2010 17:08 UTC ''Note: If two curvilinear datasets are on the same XY grid (or comparing within the same dataset), then we should do all calculations and visualizations on the native curvilinear grid of that data set. Faster, more reliable, and better looking results. So this discussion refers only to regridding needed what the source grids differ.''

We will not be able to regrid directly from one curvilinear grid to another for some time. We should eventually get that capability for version 2 of the GRIDSPEC materials in libcf, but folks are still pre-alpha on version 1. So for the foreseeable future, computing differences between two curvilinear datasets that are on differing grids will require a couple of new bits of logic in LAS.

LAS will need to ask for the variables from each dataset to be regridded to the same rectilinear grid. Ferret will then do the difference calculation and visualization on that grid. At best the regridding will always be imperfect, since curvilinear grids exist for the precise reason of working around the inadequacies of rectilinear grids. But I think we can create a functioning differencing system at modest effort.

Questions we need to answer include:

1. the logic to use in determining the rectilinear grid
2. what piece of software will perform this logic?

I think the answer to #2 has to be LAS Java code -- extending the code that currently generates the regridding request to F-TDS. The enhanced version of this code LAS will need to generate commands like the following and include these in the URL created for F-TDS (schematic syntax).

define axis/x=lox:hix:delta_x  LAS_xtarget
define axis/y=loy:hiy:delta_y  LAS_ytarget
let v1_regrid = v1[gx=LAS_xtarget, gy=LAS_ytarget]

o lox, hix, loy, hiy are simply the limits of the requested region

o delta_x, delta_y should be specifiable in the LAS request (under the end users control through the user interface), but determined by LAS if not specified in the request.

A default algorithm for determining delta_len (the shared default for delta_x and delta_y) might be this:

area = (hix-lox)*(hiy-loy)
nominal_len = SQRT(area)
delta_len = nominal_len / 200  ! target concept is a 200x200 square grid

Migrated-From: http://dunkel.pmel.noaa.gov/trac/las/ticket/789

[karlmsmith]

Comment by steven.c.hankin on 7 May 2010 22:04 UTC Well, Ansley pointed out that I had neglected to take into consideration the pre-computed "index map" files. (What a dunder head!) So here goes again -- replacing the text above with a new straw man plan....

As a matter of compute efficiency we can only regrid from a curvilinear variable to a '''pre-determined''' rectilinear grid. Furthermore a significant computation is needed to create an index map (which is a named file) for that pre-determined grid before any regridding can be performed. This leads to an entirely different strategy than what is outlined above.

When configuring a new curvilinear dataset into LAS we should:

1. determine one or more target rectilinear grids for it
2. compute the index map for each target grid and output it to a particular file
3. configure into F-TDS a virtual dataset that corresponds to each rectilinear regrid of the dataset
4. presumably configure these virtual datasets as LAS XML datasets, too
5. Add to the LAS dataset XML enough metadata (parameter/value pairs) so it "knows" the relationship between the native curvilinear dataset and its rectilinear children

The parameter value pairs might consist of (schematically) o rectilinear_regrid_default = <child dataset 1>[[BR]] o rectilinear_regrid_hires = <child dataset 2>[[BR]] o rectilinear_regrid_lores = <child dataset 3>

The logic in the LAS Java code that currently generates the regridding request to F-TDS will need to be extended so that it uses rectilinear child datasets instead of the native curvilinear datasets. The intent of this approach is to reduce the differencing of curvilinear datasets to the already solved problem of differencing rectilinear datasets.

'''Presumably the first task in implementing this strategy is to manually configure a child, rectilinear dataset into F-TDS and test it as a "standard" LAS dataset.''' Limiting the logic to support only a single (default) rectilinear dataset child of each curvilinear dataset seems adequate for the first implementation.

[karlmsmith]

Modified by @noaaroland on 7 Jan 2011 01:08 UTC