Detect when old and new grid don't overlap?

[leifdenby]

I was getting a strange error when initiating a regridder (from PET0.ESMF_LogFile):

20180926 112440.533 ERROR            PET0 ESMF_IOScrip.F90:211 ESMF_OutputWeightFile Operation not yet supported  - "factorList" has size 0 and PET count is 1. There is nothing to write.
20180926 112440.534 ERROR            PET0 ESMF_IOScrip.F90:136 ESMF_SparseMatrixWrite Operation not yet supported  - Internal subroutine call returned Error
20180926 112440.534 ERROR            PET0 ESMF_Field_C.F90:1176 f_esmf_regridstorefile Operation not yet supported  - Internal subroutine call returned Error

After some messing around I realised it was because I had written my new grid incorrectly and so the error above (There is nothing to write) makes sense :smile: I was wondering if it might be worth adding a check for this to provide a better error message than:

ESMC_FieldRegridStoreFile() failed with rc = 18.    Please check the log files (named "*ESMF_LogFile").

as that could save other people some time too.

I'm happy to add this and create a pull request, but I was just wondering whether you think this would be a good idea and where you would put this check (for example read the log file from ESMF/use something existing routine inside if esmfpy/check the bounds in xesmf when creating the regridder)?

[JiaweiZhuang]

Thanks for reporting! That's a great point. You are right that ESMF's error message is sort of hard to decipher.

where you would put this check (for example read the log file from ESMF/use something existing routine inside if esmfpy/check the bounds in xesmf when creating the regridder)?

I actually added explicit checking at several places in xesmf (at Python level, i.e. the latter approach), to prevent running into ESMF error; one common error is calling conservative regridding when the boundaries are not defined.

But your case is tricker that performing explicit checking in Python needs fairly complicated code. Just a sanity check like lat1.min() < lat2.max() is easy, but scanning for overlaps should actually be part of regridding, not preprocessing... In this case, loading the LogFile (the former approach) probably makes more sense.

One thing I need to confirm (before PR or code change) is whether nothing to write is specific to writing offline weight files. Maybe this error won't not exist with online weight retrieve (#11), say if an empty numpy array is returned in the case of no overlapping cells. @bekozi can probably answer this better 😀

[bekozi]

@leifdenby I'm assuming the unmapped action is set to ignore?

Maybe this error won't not exist with online weight retrieval (#11), say if an empty numpy array is returned in the case of no overlapping cells.

I'm not sure what will happen with disjoint geometries and online weight retrieval...will have to test. Returning zero weights makes sense, but it is normal ESMF behavior to bail out with no geometry overlap. We have an open ESMF internal ticket to better handle the logging for this case. There is talk about implementing a costly grid validation option that could eventually be useful here.

But your case is trickier that performing explicit checking in Python needs fairly complicated code.

We implemented this in ocgis, and it does get mildly complex. Disjoint checks need to account for regridding type (centers v corners), coordinate systems, longitude wrapping (-180/180 v 0/360), and masking (not sure we actually account for this).

for example read the log file from ESMF

We have talked about implementing something like this. It might be nice to simply read the log output into an exception string. Ideally, ESMPy would capture the ESMF return code and map it to a user-friendly exception, but we are not there yet.

[JiaweiZhuang]

I'm assuming the unmapped action is set to ignore?

I set unmapped_action=ESMF.UnmappedAction.IGNORE by default. ESMF.UnmappedAction.ERROR requires the target grid to be fully covered by the source grid, which is generally not true. It will indeed throw an error for disjoint grids, but will also throw an error for other reasonable cases like when the target grid is larger than the source grid.

[bekozi]

I wanted to quickly follow-up on this just in case. Are you waiting on anything from me?

The choice of unmapped action flag is up to the client of course, and I can see the logic for xesmf using the IGNORE flag. My understanding is that the default ERROR setting in ESMF was chosen because the original global modeling grids were expected to have matching spatial domains.