UXARRAY / uxarray

Xarray extension for unstructured climate and global weather data analysis and visualization.
https://uxarray.readthedocs.io/
Apache License 2.0
156 stars 32 forks source link

Negative values in some connectivity arrays breaking UXarray #481

Open erogluorhan opened 1 year ago

erogluorhan commented 1 year ago

Describe the bug Some datasets might have negative index values in their connectivity array (likely for NaN representation?), e.g. "https://web.lcrc.anl.gov/public/e3sm/inputdata/share/meshes/mpas/ocean/oQU480.230422.nc" as shown in the example below, which can break the code.

To Reproduce Steps to reproduce the behavior:

  1. Open the dataset
    
    import requests

small_file_480km = requests.get("https://web.lcrc.anl.gov/public/e3sm/inputdata/share/meshes/mpas/ocean/oQU480.230422.nc").content

ds_small_480km = ux.open_dataset(small_file_480km, small_file_480km)


2. Look into the minimum value of `ds_small_480km.uxgrid.Mesh2_node_faces`

**Expected behavior**
Such values should be properly read in (i.e. np.nan etc.) when digesting the dataset.
philipc2 commented 1 year ago

@erogluorhan

Thank you for pointing this out! Can you explain further what the concern in here? ds_small_480km.uxgrid.Mesh2_node_faces is actually computed internally, not currently parsed from MPAS datasets (#455) is planning to include direct parsing of it.

Additionally, the min value that I am getting is -FILL_VALUE, which is expected.

From my understanding, MPAS represents missing values using 0

When this occurs, connectivity lists (for example cellsOnCell) need to be updated to account for this missing element. In this case, the entry in the connectivity list should be replaced by a value of 0.

erogluorhan commented 1 year ago

Thank you for pointing this out! Can you explain further what the concern in here? ds_small_480km.uxgrid.Mesh2_node_faces is actually computed internally, not currently parsed from MPAS datasets (#455) is planning to include direct parsing of it.

Additionally, the min value that I am getting is -FILL_VALUE, which is expected.

Oops, my bad, thanks for the clarification! One more comment though: We might have forgotten to reomovegrid/utils.py/_replace_fill_values(). It looks like it is not used, and we are actually using grid/connectivity.py/_replace_fill_values()