Enhance Point2Grid to support a wider variety of input tripolar datasets

hsoh-u commented 3 months ago

Replace italics below with details for this issue.

Describe the Enhancement

point2grid recognizes a CICE tripolar data as CF compliant NetCDF, but not as a point data. If the NC-CF file is not recognized as a point data, there is a sanity checking (evenly distributed latitude and longitude). The sanity checking prevents from handling a CICE tripolar data by point2grid.

point2grid recognizes CF compliant NetCDF file with 2D lat/lon as point data for following cases:

by checking standard_name attribute (UK tripolar data)
by checking units and axis/_CoordinateAxisType attribute

The enhancements are:

check the coordinates attribute.
make the latitude and longitude variables names configurable at Point2GridConfig file (with -config option). This is a manual step.

checking coordinates attributes is not simple because a CICE tripolar data has multiple coordinates attribute values:

        float dxe(nj, ni) ;
                dxe:long_name = "E cell width through middle" ;
                dxe:units = "m" ;
                dxe:coordinates = "ELON ELAT" ;
                dxe:missing_value = 1.e+30f ;
                dxe:_FillValue = 1.e+30f ;
        float ANGLE(nj, ni) ;
                ANGLE:long_name = "angle grid makes with latitude line on U grid" ;
                ANGLE:units = "radians" ;
                ANGLE:coordinates = "ULON ULAT" ;
                ANGLE:missing_value = 1.e+30f ;
                ANGLE:_FillValue = 1.e+30f ;
        float ANGLET(nj, ni) ;
                ANGLET:long_name = "angle grid makes with latitude line on T grid" ;
                ANGLET:units = "radians" ;
                ANGLET:coordinates = "TLON TLAT" ;
                ANGLET:missing_value = 1.e+30f ;
                ANGLET:_FillValue = 1.e+30f ;
        float hi_d(time, nj, ni) ;
                hi_d:units = "m" ;
                hi_d:long_name = "grid cell mean ice thickness" ;
                hi_d:coordinates = "TLON TLAT time" ;
                hi_d:cell_measures = "area: tarea" ;
                hi_d:comment = "ice volume per unit grid cell area" ;
                hi_d:missing_value = 1.e+30f ;
                hi_d:_FillValue = 1.e+30f ;
                hi_d:cell_methods = "time: mean" ;
                hi_d:time_rep = "averaged" ;

The location of CICE tripolar data: seneca:/d1/personal/jopatz/workbench/tripolar/iceh.2018-01-03.c00.nc

Time Estimate

Estimate the amount of work required here. 2 or 3 days of work.

Sub-Issues

Consider breaking the enhancement down into sub-issues.

[ ] Add a checkbox for each sub-issue here.

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

Define the source of funding and account keys here or state NONE.

Define the Metadata

Assignee

[ ] Select engineer(s) or no engineer required
[ ] Select scientist(s) or no scientist required

Labels

[ ] Review default alert labels
[ ] Select component(s)
[ ] Select priority
[ ] Select requestor(s)

Milestone and Projects

[ ] Select Milestone as the next official version or Backlog of Development Ideas
[ ] For the next official version, select the MET-X.Y.Z Development project

Define Related Issue(s)

Consider the impact to the other METplus components.

[ ] METplus, MET, METdataio, METviewer, METexpress, METcalcpy, METplotpy

Enhancement Checklist

See the METplus Workflow for details.

[ ] Complete the issue definition above, including the Time Estimate and Funding Source.
[ ] Fork this repository or create a branch of develop. Branch name: feature_<Issue Number>_<Description>
[ ] Complete the development and test your changes.
[ ] Add/update log messages for easier debugging.
[ ] Add/update unit tests.
[ ] Add/update documentation.
[ ] Push local changes to GitHub.
[ ] Submit a pull request to merge into develop. Pull request: feature <Issue Number> <Description>
[ ] Define the pull request metadata, as permissions allow. Select: Reviewer(s) and Development issue Select: Milestone as the next official version Select: MET-X.Y.Z Development project for development toward the next official release
[ ] Iterate until the reviewer(s) accept and merge your changes.
[ ] Delete your fork or branch.
[ ] Close this issue.

j-opatz commented 3 months ago

After checking NOAA's RTOFS and GFSv17 ocean datasets, there are some notes:

RTOFS appears to be CF-compliant and does not need to have additional development in order to utilize point2grid's interpolation methods

GFSv17 does not appear to be in CF-compliant format; a call to point2grid produces the following error:

ERROR  : 
ERROR  : process_data_file() -> "/d1/personal/jopatz/workbench/tripolar/GFS_ocean/gfs.ocean.t00z.6hr_avg.f006.nc" not a valid data file
ERROR  :

I've taken these files, as well as the CICE files, and copied them over to /d1/projects/MET/MET_issues/feature_2857 on Seneca. There are subdirectories for each of the tripolar data sources.

A note on this is that every tripolar dataset we've been handed for testing has had a slightly different approach to file structure: a very simple example of this is that the CICE file uses the TLAT, TLON, ULAT, ULON, etc. for its variable coordinate attribute while the GFS data does not. So a "one size fits all" solution on this, such as checking the coordinate variable attribute might not work as previously expected.

j-opatz commented 1 month ago

For work completed so far on this issue:

lat/lon variables comes from "coordinates" attribute for CICE tripolar data are now recognized. point2grid supports multiple variables with different lat/lon variables, too.

hi_d:coordinates = "TLON TLAT time"
uvel_d:coordinates = "ULON ULAT time"

As an example, the following option is supported by reading TLON and TLAT variables for hi_d varibale, and ULON and ULAT variables for uvel_d.

-field 'name="uvel_d"; level="(0,*,*)";' -field 'name="hi_d"; level="(0,*,*)";'

One issue is not resolved. Howard is working on the lat/lon variable names from the configuration file. This is for when point2grid can not identify the lat/lon variables. User provides the lat/ lon variable names at the user defined config file. point2gird won't find the lat/lon variables in this case.

jprestop commented 3 weeks ago

@hsoh-u or @j-opatz Can you please provide a status update and let me know if any more work is needed on this before the 12.0.0-beta5 release? Or should this be moved to beta6?

dtcenter / MET