Purpose

This PR adds capacity to use a model 'mask' to identify whether individual grid cells are within the model domain. This PR also adds an example gridded attributes NetCDF file that contains a model mask and all other gridded model attributes needed to execute the model for an example test basin.

Additions

This PR adds the variable mask to domaingrid_type. The mask variable is to be read-into the model via the gridded model attributes input file. All grid cells must have a value of 0 or 1 for mask; 0 indicates that the grid cell is not within the model domain and so simulations should not be executed for that grid cell; 1 indicates that the grid cell is within the model domain.
This PR adds /data/NorthForkGriddedAttributes.nc which contains gridded attributes for the area draining to USGS gage #06311000, named 'NORTH FORK POWDER RIVER NEAR HAZELTON, WY' in CAMELS.
This PR adds /test/analysis/get_noahom_gridded_attributes_nc_file.R which was used to create /data/NorthForkGriddedAttributes.nc. /test/analysis/get_noahom_gridded_attributes_nc_file.R contains an R script that takes a hydrofabric geopackage as an input and queries Lynker AWS resources to return a NetCDF file containing the gridded model attributes (e.g., land use, soils). The gridded model attributes hosted on Lynker's AWS bucket are from NWM 2.x.
This PR adds /test/analysis/NorthFork.gpkg which is a subset of the CONUS hydrofabric pertaining to the North Fork test basin. /test/analysis/NorthFork.gpkg was used as input to /test/analysis/get_noahom_gridded_attributes_nc_file.R in creating /data/NorthForkGriddedAttributes.nc

Changes

/run/namelist.input was amended to point to /data/NorthForkGriddedAttributes.nc
/bmi/bmi_noahowp.f90: PR #86 added unit conversions for ETRAN, QSEVA, EVAPOTRANS, and ECAN, in the variables' respective listings in noahowp_set_float and noahowp_get_float. These conversions were accomplished by multiplying each variable's respective grid by a unit conversion factor (e.g., 1000 to convert m to mm). However, grid cells outside the model domain (i.e., where the mask = 0) were initialized as having values of huge(1.) and multiplying variable values for these cells (i.e., with a value of huge(1)) by unit conversion factors was causing float / overflow exceptions after compiling and running /test/noahowp_driver_test.f90. This PR adds a temporary/conversion grid to solve this problem. Note that the arithmetic exception did not occur on all of these variables but I implemented the fix for all because I felt it would be better to never try an arithmetic operation outside the model mask.
src/AttributesType.f90: Was amended to include the new model mask as a 2D integer variable. Additionally, this PR removes the requirement for dx and dy as NetCDF-defined attributes to be included in the input NetCDF attributes nc file. Instead, the code base will infer dx and dy from the longitude (x) and latitude (y) NetCDF-defined dimensions/variables.
/data/netcdf_input.nc was removed as it was intended to be replaced by /data/NorthForkGriddedAttributes.nc
/test/analysis/create_dummy_netcdf.py and /test/analysis/create_dummy_netcdf_singlefile.py were removed because they were used to create dummy gridded attribute files and are obsolete following the creation of /test/analysis/get_noahom_gridded_attributes_nc_file.R

Testing

Successfully compiled and executed /driver/NoahModularDriver.f90 and /test/noahowp_driver_test.f90
The shell command ncdump -h NorthForkGriddedAttributes.nc (executed within /data/) gave this output.

Notes

Soil color is a required gridded model input variable/attribute but was not available via the Lynker AWS bucket, which /test/analysis/get_noahom_gridded_attributes_nc_file.R queries to obtain all other model attributes. The Noah-MP user guide states that, "for a case lack of soil color data, one may choose a medium dark color index (= 4 or 5). For a sandy soil or desert, it is better to choose the lightest index (=1)" (pg. 15). A soil color value of 5 was assumed across the entire model domain for the North Fork test basin.

@drakest123 can you clarify what you mean by NorthForkGriddedAttributes.nc is not uniform rectilinear? My understanding is that dx and dy are constants and in degrees lat and long. Are you saying that dx and dy shouldn't be constants if they are provided in degrees lat and long?

The North Fork grid should correspond to the NWM 3 grid, which I thought was uniform rectilinear.

@drakest123 I think I understand after reading your email to Scott that you cc'd me on. I think you're saying it can't be uniform rectilinear because the dx and dy are constant degrees lat and long and that when projected, they won't be uniform.

I wonder if this problem might be solved by amending the code to expect lat and long variables but not expect that the grid's dimensions are in lat and long? The NWM gridded resources (see ngen_gridded_data.nc) are provided in lambert conformal but I converted the gridded resources dimensions to be degrees lat and long because the model requires that information for its simulation. I could instead leave the dimensions as they are and add lat and long variables.

Hi Grey,

I’d like to get Scott’s input on this in case I am misinterpreting the how BMI interfaces with the gridded data functionality of a model. It is key to determining which BMI related functions need to be coded.

I looked for but did not see “ngen_gridded_data.nc” in the noah-owp-modular directory structure. Should I be looking in a different place? In either case, lambert conformal is a conic projection so it would not be uniform rectilinear either.

I could be wrong about all of this but it seems to me that from the reference: https://bmi.readthedocs.io/en/stable/model_grids.html#model-grids that if the model runs on a uniform rectilinear grid then the BMI functions that need to be implemented are: get_grid_rank https://bmi.readthedocs.io/en/stable/index.html#get-grid-rank get_grid_size https://bmi.readthedocs.io/en/stable/index.html#get-grid-size get_grid_shape https://bmi.readthedocs.io/en/stable/index.html#get-grid-shape get_grid_spacing https://bmi.readthedocs.io/en/stable/index.html#get-grid-spacing get_grid_origin https://bmi.readthedocs.io/en/stable/index.html#get-grid-origin

And if the model runs on a rectilinear grid then the BMI functions that need to be implemented are:

get_grid_rank https://bmi.readthedocs.io/en/stable/index.html#get-grid-rank get_grid_size https://bmi.readthedocs.io/en/stable/index.html#get-grid-size get_grid_shape https://bmi.readthedocs.io/en/stable/index.html#get-grid-shape get_grid_x https://bmi.readthedocs.io/en/stable/index.html#get-grid-x get_grid_y https://bmi.readthedocs.io/en/stable/index.html#get-grid-y get_grid_z https://bmi.readthedocs.io/en/stable/index.html#get-grid-z

Regards, Steve

On Feb 28, 2024, at 5:47 AM, GreyEvenson-NOAA @.***> wrote:

@drakest123 https://github.com/drakest123 I think I understand after reading your email to Scott that you cc'd me on. I think you're saying it can't be uniform rectilinear because the dx and dy are constant degrees lat and long and that when projected, they won't be uniform.

I wonder if this problem might be solved by amending the code to expect lat and long variables but not expect that the grid's dimensions are in lat and long? The NWM gridded resources https://www.lynker-spatial.com/#gridded-resources/ (see ngen_gridded_data.nc) are provided in lambert conformal but I converted the gridded resources dimensions to be degrees lat and long because the model requires that information for its simulation. I could instead leave the dimensions as they are and add lat and long variables.

— Reply to this email directly, view it on GitHub https://github.com/NOAA-OWP/noah-owp-modular/pull/92#issuecomment-1969021953, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE5J22B3Y6KBRTWZ5CWYRSLYV4YOVAVCNFSM6AAAAAA645RTFWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRZGAZDCOJVGM. You are receiving this because you were mentioned.

Hi @drakest123,

Sounds good. I'll look for Scott's response but just to clarify, we can assume uniform rectilinear so we don't need to worry about the get_grid_x, etc subroutines. However, I do think we need to worry about getting /data/NorthForkGriddedAttributes.nc into an appropriate format/projection so that it reflects a uniform rectilinear grid.

Can I ask: what format/projection should the input attribute data be in for it to be uniform rectilinear? Because /data/NorthForkGriddedAttributes.nc is made by test/analysis/get_noahom_gridded_attributes_nc_file.R by subsetting ngen_gridded_data.nc variables, we can project the lambert conformal NWM grids to suite our needs.

The NWM grids are in Lynker's AWS bucket here and ngen_gridded_data.nc is in the gridded-resources subdirectory.

If my understanding of uniform rectilinear is correct then UTM is an example. It looks like all of the data points in your data/NorthForkGriddedAttributes.nc example are in UTM zone 13T. Since they don’t span multiple UTM zones you should not have to deal with the complication of multiple UTM zones when interpolating data between coordinate systems.

I’d like to discuss this on Thursday if you are available. I’m mostly available after 9AM PST.

On Feb 28, 2024, at 12:04 PM, GreyEvenson-NOAA @.***> wrote:

Hi @drakest123 https://github.com/drakest123,

Sounds good. I'll look for Scott's response but just to clarify, we can assume uniform rectilinear so we don't need to worry about the get_grid_x, etc subroutines. However, I do think we need to worry about getting /data/NorthForkGriddedAttributes.nc into an appropriate format/projection so that it reflects a uniform rectilinear grid.

Can I ask: what format/projection should the input attribute data be in for it to be uniform rectilinear? Because /data/NorthForkGriddedAttributes.nc is made by test/analysis/get_noahom_gridded_attributes_nc_file.R by subsetting ngen_gridded_data.nc variables, we can project the lambert conformal NWM grids to suite our purposes.

The NWM grids are in Lynker's AWS bucket here https://www.lynker-spatial.com/ and ngen_gridded_data.nc is in the gridded-resources subdirectory.

— Reply to this email directly, view it on GitHub https://github.com/NOAA-OWP/noah-owp-modular/pull/92#issuecomment-1969776707, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE5J22HMTGK5CB3X6TGKSELYV6EWRAVCNFSM6AAAAAA645RTFWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRZG43TMNZQG4. You are receiving this because you were mentioned.

@drakest123 Sounds good. I'm good to meet 9-130, 330-5 EST. I'll look for a call invite from you.

I'll admit, I'm not quite following why UTM would be uniform rectilinear while other projections would not. It seems to me that given a grid origin and if dx and dy are constant linear distances then we should get uniform rectilinear grid. That said, I can understand why the lat and long values would not be constant across the whole of the grid, given the distortion of projections. Looking forward to talking about this!

Hi Guys, Sorry for the slow reply. I just had a minor surgery on my hand. But I'd be happy to join a call on this. I could maybe do a late afternoon call today, but tomorrow would work better. Best regards, Scott

On Thu, Feb 29, 2024 at 6:36 AM GreyEvenson-NOAA @.***> wrote:

@drakest123 https://github.com/drakest123 Sounds good. I'm good to meet 9-130, 330-5 EST. I'll look for a call invite from you.

I'll admit, I'm not quite following why UTM would be uniform rectilinear while other projections would not. It seems to me that given a grid origin and if dx and dy are constant linear distances then we should get uniform rectilinear grid. That said, I can understand why the lat and long values would not be constant across the whole of the grid, given the distortion of projections. Looking forward to talking about this!

— Reply to this email directly, view it on GitHub https://github.com/NOAA-OWP/noah-owp-modular/pull/92#issuecomment-1971158967, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB2T4LY3D5HUPTGJGNXEPYTYV4XGHAVCNFSM6AAAAAA645RTFWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZRGE2TQOJWG4 . You are receiving this because you are subscribed to this thread.Message ID: @.***>

Sorry about your hand, Scott. My schedule is fairly open on Friday so I’ll check with Grey on his availability.