Closed GeorgeGayno-NOAA closed 4 years ago
Note to self: The 'emcsfc' builds use v2.5 of the NCEP g2 library. Under /apps/contrib/NCEPLIBS/orion/modulefiles, there is only v3.2.0. But I could not get emcsfc to compile without this error: _undefined reference to `png_set_longjmpfn
According to George V.: "This is an incompatibility between our version of png in NCEPLIBS and the version on the system. Some of the g2 build is picking up our version and some, the system's."
So I switched to George V's version of g2 v2.5 here: /apps/contrib/NCEPLIBS/lib/modulefiles to get emcsfc to compile.
Also, if I use George V's g2 v3 and his version of png, it compiles without error. Will contact the NCEPLIBS group.
All programs within the repository were successfully compiled on Orion (ffab5c4).
Next, run any unit and regression tests. Note: there is no test for the 'nst_tf_chg' program, which is out-of-date (#90)
The nemsio utilities - nemsio_get
, nemsio_chgdate
, nemsio_read
, mkgdsnemsioctl
- were tested using the script and data in: /work/noaa/da/ggayno/save/ufs_utils.git/orion_port/nemsio_utils
The specific tests are:
nemsio_get
to print header information from a nemsio file.nemsio_chgdate
to change the date stamp in a nemsio file.nemsio_read
to dump the contents of a nemsio file.mkgfsnemsioctl
to create a Grads control file for viewing a nemsio file.The nemsio test was run using ffab5c4 of the feature/orion branch. Output from the first three utilities was stored to this log file: /work/noaa/da/ggayno/save/ufs_utils.git/orion_port/nemsio_utils/log.unit.test. All output looked correct. The Grads control file from the last utility was stored in the same directory: gdas.t00z.sfcanl.nemsio.ctl. The file was visualized using that control file and all fields displayed correctly.
Next, the grid generation was tested. This includes the fre-nctools
, orog
and sfc_climo_gen
programs. Using ffab5c4, C96 uniform, C96 stretched, C96 nested and C96 regional grids were created using the driver_grid.orion.sh
script in ./UFS_UTILS/driver_scripts. The default grid specs in the script were not modified.
All grids were successfully created. The 'grid', 'orog' and 'sfc_climo' files were visualized using ncview
and looked correct.
Next, the snow2mdl
program was tested using the script and data in /work/noaa/da/ggayno/save/ufs_utils.git/orion_port/snow2mdl. The test (using ffab5c4) blends IMS and AFWA data to create a T1534 global analysis of snow depth and cover (grib1 format).
The test ran successfully and the output file was visualized in Grads and looked correct. The same test was run on Hera using 2d7f796 of 'develop'. The output files were bit identical.
Next, the global_cycle
program was tested (ffab5c4) using a canned case. The run script and data are here: /work/noaa/da/ggayno/save/ufs_utils.git/orion_port/cycle. The test updates the surface fields for a C768 uniform global grid. The same test was run on Hera, using 'develop' at 2d7f796. The output tiled netCDF files from both tests were compared using the nccmp
utility and were identical.
Next, the serial version of chgres
was tested (the branch at ffab5c4) using a canned case: /work/noaa/da/ggayno/save/ufs_utils.git/orion_port/chgres_serial. The test converts GFS gaussian nemsio files to C96 global uniform. The same test was run on Hera, using 'develop' at 2d7f796. chgres
outputs atmospheric and surface tiled netCDF files. The files from both tests were identical according to the nccmp
utility.
Next, the emcsfc_ice_blend
program was tested (used the branch at ffab5c4). The test blends IMS ice cover and EMC ice concentration data. The output blended file is 1/12-degree, grib 1 format. The script and data for the canned case are here: /work/noaa/da/ggayno/save/ufs_utils.git/orion_port/ice_blend. The same test was run on Hera using 'develop' at 2d7f796. The files from both tests were bit-identical.
Note: the ice blend process uses copygb to replace the bitmap with a flag value of 1.57 in the final file. This version of copygb gave a wrong answer: /apps/contrib/NCEPLIBS/orion/utils/grib_util.v1.2.0/exec/copygb. This version gave the correct answer: /apps/contrib/NCEPLIBS/lib/NCEPLIBS-grib_util/v1.1.1/exec/copygb. Will notify the developer of this. In summary, the ice_blend
program is working on Orion.
Finally, the chgres_cube
regression test suite was run (using ffab5c4). The baseline comparison data was copied from Hera. All tests passed:
<<< C192 FV3 HISTORY TEST PASSED. >>>
<<< C96 FV3 GAUSSIAN NEMSIO TEST PASSED. >>>
<<< C96 GFS SIGIO TEST PASSED. >>>
<<< C96 GFS GAUSSIAN NEMSIO TEST PASSED. >>>
<<< C96 REGIONAL TEST PASSED. >>>
<<< C192 GFS GRIB2 TEST PASSED. >>>```
All programs are working on Orion. One final task is to the test the ./utils/gdas_init utility. However, it requires access to HPSS. And Orion can't talk to HPSS. George V. is developing a work-around. Standing by.
There are issues building g2 with png support. PNG 1.2.44 (and maybe others) has a header with different structure from earlier and also much more recent PNG releases. If different releases are used when the png library is built and when g2 is linked, there will be an unresolved png-longjmp external. This was a nasty problem for me for some weeks in 2017 getting NCEPLIBS to build on newer theia releases before I found it.
The solution is to do one of two things.
Use png/v1.2.59 along with g2.
When building the g2 library make sure that the png includes are NOT from the linux distribution ones in /usr/include or /usr/local/include or /opt/include. SPECIFY them by loading the png module used in post and other app builds.
When I build my G2, these are set
export PNG_INC=$NCEPLIBS/EXTERNAL/LOCAL_EXTERN/png/v1.2.44/include/
export JASPER_INC=$NCEPLIBS/EXTERNAL/LOCAL_EXTERN/jasper_v1.900.1/include export Z_INC=$NCEPLIBS/EXTERNAL/LOCAL_EXTERN/z/v1.2.6/include
(I probably need to change this last one to z1.2.7 which is generally used elsewhere)
Before merging, check with Kate about which versions of netcdf and esmf to use.
@KateFriedman-NOAA What more needs to be done for this ticket?
Nothing I can think of. I checked and see this is in the Orion build modulefile:
module use -a /apps/contrib/NCEPLIBS/lib/modulefiles
module load netcdfp/4.7.4.release
module load esmflocal/8_0_0.release
...which was the only bit I needed to check was in. I've been running a longer test on Orion since the maintenance last week ended and have no issues so we're good to finish this and merge back.
We will need to update those netcdf and esmf modules again when the updated versions are officially installed and ready. Can do a small PR for that when ready. Thanks!
Will we get a new UFS_UTILS tag afterwards?
Nothing I can think of. I checked and see this is in the Orion build modulefile:
module use -a /apps/contrib/NCEPLIBS/lib/modulefiles module load netcdfp/4.7.4.release module load esmflocal/8_0_0.release
...which was the only bit I needed to check was in. I've been running a longer test on Orion since the maintenance last week ended and have no issues so we're good to finish this and merge back.
We will need to update those netcdf and esmf modules again when the updated versions are officially installed and ready. Can do a small PR for that when ready. Thanks!
Will we get a new UFS_UTILS tag afterwards?
The GDAS initialization scripts can be added later if they ever allow Orion to access HPSS.
Once I merge, I can create a tag or you can just point to the proper snapshot of 'develop'. Or are you referring to the 'release/ops-gfsv16' branch? That release branch is several commits behind 'develop'. I did not update the release branch for Orion (but I can).
The GDAS initialization scripts can be added later if they ever allow Orion to access HPSS.
Yep...most archive/HPSS stuff will have to wait too. Sounds good, thanks!
Once I merge, I can create a tag or you can just point to the proper snapshot of 'develop'. Or are you referring to the 'release/ops-gfsv16' branch? That release branch is several commits behind 'develop'. I did not update the release branch for Orion (but I can).
We need to get Orion support into the release/ops-gfsv16 branch so yes, please update the release branch. Thanks!
The GDAS initialization scripts can be added later if they ever allow Orion to access HPSS.
Yep...most archive/HPSS stuff will have to wait too. Sounds good, thanks!
Once I merge, I can create a tag or you can just point to the proper snapshot of 'develop'. Or are you referring to the 'release/ops-gfsv16' branch? That release branch is several commits behind 'develop'. I did not update the release branch for Orion (but I can).
We need to get Orion support into the release/ops-gfsv16 branch so yes, please update the release branch. Thanks!
Kate, sorry for being asleep at the wheel. I was totally focused on the head of develop. Will try to get it done by sometime next week.
No worries and thanks! :)
I will work the Orion port of the GFS v16 release branch under this issue #26.
The branch at e55d18e was successfully compiled on Cray, Dell, Hera, Orion and Jet. All regression tests passed on those machines. Will merge.
@KateFriedman-NOAA This was merged today. I will keep the 'feature/orion' branch available for a few more days. Then delete it.
@GeorgeGayno-NOAA Great, thanks!
Deleting branch and closing issue.
Port the repo to the new Orion machine.
Also, remove all references to WCOSS-Phase 1/2, which is being turned off by the end of April.