Extracting a mooring via subsample.mooring_array, first calls subsample.cutout, for arbitrary datasets. In the case of ECCO or LLC4320, subsample.cutout is a necessary intermediate step that "removes" face as a dimension by "transforming" the surviving subdomain of the original dataset. By "transforming", I mean reducing the horizontal dimensional of the dataset from (face, Y, X) to (Y, X). Given the complex topology of the dataset (e.g., increasing the index value along the local X axis within the faces={7, 8, 9, 10, 11, 12} samples data that lies southward instead of Eastward), the "transformation" requires that a subset of the entire dataset (of arbitrary size) to be transposed and their "ordering" reversed (plus other label manipulations).
The transformation (removal of face) is quite fast for the ECCO dataset but can be very computationally expensive for larger datasets (e.g. DYAMOND or LLC4320). If only a 1d-like array is the final desired output, like in the case of subsample.mooring_array, then the "current order of operations" is a bit wasteful even though it is all done lazily. Such "expensive" feature can manifest itself by the creation of a large number of task graphs required in the intermediate step of transforming the data, which in many cases can overwhelm the system (kernel dies). Furthermore, the larger the extend of the array pathway in lat/lon space (i.e. spanning multiple faces) implies a larger dataset that first needs to be "transformed". Consider the example below:
a cutout with this coordinates involves the faces [0, 1, 2, 3, 4, 5, 7, 8, 9, 10, 11, 12] and results in the following cutout plot
From the figure, about half of the ~ 300 X 180 (50869) grid points needs to be "transformed" (i.e. transposed and their ordering reversed), but only about 1000 points are retained by the final mooring array, i.e. len(od_moor.dataset.mooring.values), and only about half of those need to be "transformed". Including the vertical and time dimensions increases very rapidly the amount of data that needs to be "transformed" while the amount of data retained grows much much slowly.
Proposed optional behavior
Include a serialized approach to subsample.mooring_array, so that a mooring array within each face is first calculated without any subsample.cutout call, and then have these arrays combined within a single array along the new dimension mooring. This (serialization) feature will be an option, but in the case face is a dimension of the dataset, this should be the default.
In the example above, the figure below color codes data within a face along the mooring dimension
Different colors along the mooring dimension can represent the same face, sampled "later" in the iterative (looped) process. This approach can then be delayed (parallelized).
This proposed approach also prevents the presence of NAN-ed data in the coordinate variables, a feature that began to produce failing tests as explained in #378 with the newer version of scipy.
Mikejmnez
commented
8 months ago
Description of current behavior
Extracting a mooring via
subsample.mooring_array, first callssubsample.cutout, for arbitrary datasets. In the case of ECCO or LLC4320,subsample.cutoutis a necessary intermediate step that "removes" face as a dimension by "transforming" the surviving subdomain of the original dataset. By "transforming", I mean reducing the horizontal dimensional of the dataset from(face, Y, X)to(Y, X). Given the complex topology of the dataset (e.g., increasing the index value along the localXaxis within the faces={7, 8, 9, 10, 11, 12} samples data that lies southward instead of Eastward), the "transformation" requires that a subset of the entire dataset (of arbitrary size) to be transposed and their "ordering" reversed (plus other label manipulations).The transformation (removal of
face) is quite fast for the ECCO dataset but can be very computationally expensive for larger datasets (e.g. DYAMOND or LLC4320). If only a 1d-like array is the final desired output, like in the case ofsubsample.mooring_array, then the "current order of operations" is a bit wasteful even though it is all done lazily. Such "expensive" feature can manifest itself by the creation of a large number of task graphs required in the intermediate step of transforming the data, which in many cases can overwhelm the system (kernel dies). Furthermore, the larger the extend of the array pathway in lat/lon space (i.e. spanning multiple faces) implies a larger dataset that first needs to be "transformed". Consider the example below:"wasteful" example:
Visually, these coordinates are:
a cutout with this coordinates involves the faces
[0, 1, 2, 3, 4, 5, 7, 8, 9, 10, 11, 12]and results in the following cutout plotFrom the figure, about half of the ~ 300 X 180 (50869) grid points needs to be "transformed" (i.e. transposed and their ordering reversed), but only about 1000 points are retained by the final mooring array, i.e.
len(od_moor.dataset.mooring.values), and only about half of those need to be "transformed". Including the vertical and time dimensions increases very rapidly the amount of data that needs to be "transformed" while the amount of data retained grows much much slowly.Proposed optional behavior
Include a serialized approach to
subsample.mooring_array, so that a mooring array within each face is first calculated without anysubsample.cutoutcall, and then have these arrays combined within a single array along the new dimensionmooring. This (serialization) feature will be an option, but in the casefaceis a dimension of the dataset, this should be the default.In the example above, the figure below color codes data within a face along the mooring dimension
Different colors along the mooring dimension can represent the same face, sampled "later" in the iterative (looped) process. This approach can then be delayed (parallelized).
This proposed approach also prevents the presence of NAN-ed data in the coordinate variables, a feature that began to produce failing tests as explained in #378 with the newer version of scipy.