Obtain new CMIP6 grids and add to data test suite

[durack1]

It occurred to me that there are likely a bunch of new (weird) grids in the pipeline for CMIP6 (particularly the ocean data) and consequently it would be a good idea to get access to this data and add these grids to the current UV-CDAT devel test suite, and additionally feed these new specs/files through to the ESMF folks so that their software updates can deal with the new formats.

If this did work, then it would mean that UV-CDAT 2.x/3.x would automagically be able to happily regrid and read all the as yet unreleased data formats.. And we'd certainly be ahead of the game when the CMIP6 data starts appearing on ESGF rather than playing catchup as was done with CMIP5..

@doutriaux1 @aashish24 I'm happy for you folks to assign this to me, and some future milestone.. Maybe UV-CDAT 2.5/3.x?

@williams13 - not sure what discussions you've been having with Balaji about the gridspec stuff, but I think we know enough now to start thinking about this..

[williams13]

Agreed. However, we need to put this in perspective and add to the prioritized task list. This may not be the highest priority since we are at least a year or two away from seeing CMIP6 data. We have a number of other items to tackle before then. ;-)

From: "Paul J. Durack" notifications@github.com<mailto:notifications@github.com> Reply-To: UV-CDAT/uvcdat reply@reply.github.com<mailto:reply@reply.github.com> Date: Thursday, October 16, 2014 8:49 AM To: UV-CDAT/uvcdat uvcdat@noreply.github.com<mailto:uvcdat@noreply.github.com> Cc: Dean Williams williams13@llnl.gov<mailto:williams13@llnl.gov> Subject: [uvcdat] Obtain new CMIP6 grids and add to data test suite (#819)

It occurred to me that there are likely a bunch of new (weird) grids in the pipeline for CMIP6 (particularly the ocean data) and consequently it would be a good idea to get access to this data and add these grids to the current UV-CDAT devel test suite, and additionally feed these new specs/files through to the ESMF folks so that their software updates can deal with the new formats.

If this did work, then it would mean that UV-CDAT 2.x/3.x would automagically be able to happily regrid and read all the as yet unreleased data formats.. And we'd certainly be ahead of the game when the CMIP6 data starts appearing on ESGF rather than playing catchup as was done with CMIP5..

@doutriaux1https://github.com/doutriaux1 @aashish24https://github.com/aashish24 I'm happy for you folks to assign this to me, and some future milestone.. Maybe UV-CDAT 2.5/3.x?

@williams13https://github.com/williams13 - not sure what discussions you've been having with Balaji about the gridspec stuff, but I think we know enough now to start thinking about this..

— Reply to this email directly or view it on GitHubhttps://github.com/UV-CDAT/uvcdat/issues/819.

[durack1]

@williams13 you're right we're 1-2 years away from seeing the data on ESGF, but the models are now starting to be tuned in preparation for their final production CMIP6 runs, so by now the grids and major config is complete.. Getting a head start on this would be a really nice feature to highlight as the UV-CDAT package continues to get more and more tight, and used by more and more folks at modeling centers around the globe.. The GFDL folks are already very excited by their new found ease of regridding functionality that UV-CDAT has given them..

As I'm not formally in the dev team or funded under it, I'll just add this as a side project of mine, just as I contributed toward the libCF/ESMF integration with Alex and Dave (TechX)..

[durack1]

Folks - FYI, I think the 7.0 branch should be considered for testing in UV-CDAT 2.1+..

-----Original Message----- From: Ryan OKuinghttons Date: Thursday, November 6, 2014 at 8:29 PM To: "Paul J. Durack" Cc: Support esmf_support esmf_support@list.woc.noaa.gov Subject: Re: 3613640 Re: [esmf_support] ESMP and numpy 1.9.0

Hi Paul,

The patch you mentioned is not in the v6.3.0rp1 release, but it is available in a beta snapshot to the 7.0 release. If the UV-CDAT team would like to experiment with that, the latest snapshot can be downloaded with the following command:

git archive --remote=git://git.code.sf.net/p/esmf/esmf --format=tar --prefix=esmf/ ESMF_7_0_0_beta_snapshot_24 | tar xf -

With regard to the CMIP6 data that you mentioned, it would indeed be very helpful for us to have access to the files you mentioned for testing. We run nightly tests on all four regridding interfaces: Fortran, C, Python and shell application. If there is anything in particular that you would like to see us demonstrate with this data, please also let us know what that might be.

We would be happy to take a look at the GFDL regridding tools as well. My schedule is fairly open, with Wednesdays being the exception, so if you could please let me know a time that works for you we can start from there. Thanks!

Ryan

On Nov 4, 2014, at 10:51 AM, Durack, Paul J. wrote:

Hello again Ryan,

With regards to the numpy 1.9.x patch you noted below, is this in the ESMF v6.3.0rp1 release? If so it would be good for the UV-CDAT devels to pull this across and test in a development branch..

On another topic, I’ve become involved in some work on the upcoming CMIP6 ocean model output specification and it became apparent to me that there may be new grids and data formats in the pipeline.. Would it be helpful for the ESMF team to have access to a couple of time steps of this new data (and it’s grid info)?

I provided CMIP5 data through to Robert Oehmke and Peggy Li back in January 2012, which admittedly was quite a bit later than when the CMIP5 data began to become available.. Just wondering whether enabling you folks with new data earlier in the CMIP6 timeline would be useful?

I have also spoken with Steve Griffies who is intimately involved in the GFDL ocean model development and he indicated that GFDL has been working on regridding functionality for their upcoming CMIP6 ocean model.. Steve was more than happy to provide you/ESMF folks with the tools they’ve developed in the hope that these could aid generalization of software for the entire CMIP6 suite..

Please do let me know if you’re interested in discussing this further.

Cheers, P

[durack1]

@dnadeau4 @doutriaux1 just pinging you both on this.. @dnadeau4 I am hoping to pull these new grids in for testing when we get to tweaking new grid functionality