search

NERC-CEH / dri_gridded_data

GNU General Public License v3.0
0 stars 0 forks source link

Figure out why coordinates are having an extra dimension added #1

Open mattjbr123 opened 1 month ago

mattjbr123 commented 1 month ago 
          Successfully run with parallelisation with this environment.

Though I've now noticed that the pipeline has added a time dimension to the variables that we converted from data variables to coordinate variables in the preprocessing function (so the variables x_bnds, y_bnds and crs)

_Originally posted by @mattjbr123 in https://github.com/NERC-CEH/dri_gridded_data/issues/9

mattjbr123 commented 1 month ago

Spinning it off into it's own issue given it's a fairly minor problem at this stage

mattjbr123 commented 6 days ago

The error will likely be in the pangeo functions that wrap around Beam, so that's probably the place to start...

mattjbr123 commented 4 days ago

Following the advice here to find out which step of the pipeline the extra dimensions get added, as a starting point.

Annoyingly, a Beam print function can only be placed at the end of a Beam pipeline, (though my hunch is there is some more fancy syntax that allows for pipeline branching and works around this), so it's a case of one-by-one putting the print statement after each function and analysing the output until we see the extra dimensions added.

mattjbr123 commented 4 days ago

Looks like it's to do with the ConsolidateDimensionCoordinates() function, which I guess isn't massively surprising... Will confirm and look into exactly what it's doing next.