search

glemieux / cime

Common Infrastructure for Modeling the Earth
Other
0 stars 0 forks source link

Run regression test: http://esmci.github.io/cime/users_guide/porting-cime.html #1

Open glemieux opened 5 years ago

glemieux commented 5 years ago

was not downloading the necessary input data from cesm due to netcdf problem

glemieux commented 5 years ago

Re-installed netcdf, but CIME test is now failing due to "ERROR: CIME models require NETCDF in PIO build". This issue states that the issue is not having NETCDF-Fortran (https://github.com/ESMCI/cime/issues/2936). Switching over to building NetCDF-Fortran (#5 and #6).

glemieux commented 5 years ago

Regression test seems to be failing when looking for gmake. Didn't a document somewhere say that gmake was hardcoded into some of the routines? Was the for HDF5? Regardless I remember building (and there still exists) a symbolic gmake link to make in ./local/bin/ I'm going to try by defining the .cime/config_machines.xml to use make by adding <GMAKE>make</GMAKE>.

glemieux commented 5 years ago

Ok, so that fixed a bunch of failures. Dropped from 42 failures down to:

Ran 395 tests in 548.647s

FAILED (failures=13, skipped=18)

A bunch of the input data is not found and sounds like it's not being download. That said the first failure in that list has this in the output file:

Installing pio version 1
ERROR: CIME models require NETCDF in PIO build

This forum thread suggests that NetCDF-Fortran needs to be discoverable: https://bb.cgd.ucar.edu/pio-build-problem-0

Added the installed directory to PATH and LD_LIBRARY_PATH.

glemieux commented 5 years ago

That didn't seem to work, but the error suggests that it might be because I need to specify the C path explicitly now?

cat: Filepath: No such file or directory
/home/gregorylemieux/projects/scratch/scripts_regression_test.20190419_145826/TestCreateNewcase/testcreatenewcase/Tools/Makefile:195: *** "NETCDF_FORTRAN_PATH specified without NETCDF_C_PATH".  Stop.
ERROR: cat: Filepath: No such file or directory
glemieux commented 5 years ago

Added export NETCDF_C_PATH to my .bashrc. I'm getting the same error as before regarding the PIO build (see 2 above). Trying to add export NETCDF_PATH to the whole thing as well. If this doesn't work, I'm wondering if I should have the NetCDF-Fortran installation point to the NetCDF-C install directory as suggested here: https://www.unidata.ucar.edu/software/netcdf/docs/building_netcdf_fortran.html

glemieux commented 5 years ago

Looking at the output file again, it looks like I made a mistake in defining the fortran path variable: f951: Warning: Nonexistent include directory ‘{/home/gregorylemieux}/local/netcdf-fortran-4.4.5/include’ [-Wmissing-include-dirs] Sure enough I forgot the $ before the call to {HOME} in the variable call.

glemieux commented 5 years ago

Now I'm getting a new and more difficult to parse error via 'test_c_create_clone_keepexe'. I think I'm going to punt and submit an issue to cime.

glemieux commented 5 years ago

Submitted: https://github.com/ESMCI/cime/issues/3087

glemieux commented 5 years ago

Using ./check_input_data --download script from one of the previous failed cases returns this:

 Loading input file list: 'Buildconf/datm.input_data_list'
  Model datm missing file domain1 = '/home/gregorylemieux/projects/cesm-inputdata/atm/cam/chem/trop_mozart_aero/aero/aerosoldep_WACCM.ensmean_monthly_hist_1849-2015_0.9x1.25_CMIP6_c180926.nc'
  Model datm missing file file1 = '/home/gregorylemieux/projects/cesm-inputdata/atm/cam/chem/trop_mozart_aero/aero/aerosoldep_WACCM.ensmean_monthly_hist_1849-2015_0.9x1.25_CMIP6_c180926.nc'
  Model datm missing file domainfile = '/home/gregorylemieux/projects/cesm-inputdata/share/domains/domain.lnd.fv1.9x2.5_gx1v6.090206.nc'
  Model datm missing file factorfn = '/home/gregorylemieux/projects/cesm-inputdata/atm/datm7/CORE2/COREv2.correction_factors.T62.121007.nc'
Loading input file list: 'Buildconf/drof.input_data_list'
  Model drof missing file domain1 = '/home/gregorylemieux/projects/cesm-inputdata/lnd/dlnd7/RX1/runoff.daitren.annual.20190226.nc'
  Model drof missing file file1 = '/home/gregorylemieux/projects/cesm-inputdata/lnd/dlnd7/RX1/runoff.daitren.annual.20190226.nc'
Loading input file list: 'Buildconf/cpl.input_data_list'
  Model cpl missing file atm2ocn_fmapname = '/home/gregorylemieux/projects/cesm-inputdata/cpl/gridmaps/fv1.9x2.5/map_fv1.9x2.5_TO_gx1v6_aave.130322.nc'
  Model cpl missing file atm2ocn_smapname = '/home/gregorylemieux/projects/cesm-inputdata/cpl/gridmaps/fv1.9x2.5/map_fv1.9x2.5_TO_gx1v6_blin.130322.nc'
  Model cpl missing file atm2ocn_vmapname = '/home/gregorylemieux/projects/cesm-inputdata/cpl/gridmaps/fv1.9x2.5/map_fv1.9x2.5_TO_gx1v6_patc.130322.nc'
  Model cpl missing file ocn2atm_fmapname = '/home/gregorylemieux/projects/cesm-inputdata/cpl/gridmaps/gx1v6/map_gx1v6_TO_fv1.9x2.5_aave.130322.nc'
  Model cpl missing file ocn2atm_smapname = '/home/gregorylemieux/projects/cesm-inputdata/cpl/gridmaps/gx1v6/map_gx1v6_TO_fv1.9x2.5_aave.130322.nc'
  Model cpl missing file atm2ice_fmapname = '/home/gregorylemieux/projects/cesm-inputdata/cpl/gridmaps/fv1.9x2.5/map_fv1.9x2.5_TO_gx1v6_aave.130322.nc'
  Model cpl missing file atm2ice_smapname = '/home/gregorylemieux/projects/cesm-inputdata/cpl/gridmaps/fv1.9x2.5/map_fv1.9x2.5_TO_gx1v6_blin.130322.nc'
  Model cpl missing file atm2ice_vmapname = '/home/gregorylemieux/projects/cesm-inputdata/cpl/gridmaps/fv1.9x2.5/map_fv1.9x2.5_TO_gx1v6_patc.130322.nc'
  Model cpl missing file ice2atm_fmapname = '/home/gregorylemieux/projects/cesm-inputdata/cpl/gridmaps/gx1v6/map_gx1v6_TO_fv1.9x2.5_aave.130322.nc'
  Model cpl missing file ice2atm_smapname = '/home/gregorylemieux/projects/cesm-inputdata/cpl/gridmaps/gx1v6/map_gx1v6_TO_fv1.9x2.5_aave.130322.nc'
  Model cpl missing file rof2ocn_liq_rmapname = '/home/gregorylemieux/projects/cesm-inputdata/cpl/gridmaps/rx1/map_rx1_to_gx1v6_nn_open_ocean_nnsm_e1000r300_marginal_sea_170503.nc'
  Model cpl missing file rof2ocn_ice_rmapname = '/home/gregorylemieux/projects/cesm-inputdata/cpl/gridmaps/rx1/map_rx1_to_gx1v6_nnsm_e1000r300_170503.nc'
Loading input file list: 'Buildconf/docn.input_data_list'
  Model docn missing file domain1 = '/home/gregorylemieux/projects/cesm-inputdata/atm/cam/ocnfrac/domain.camocn.0.9x1.25_gx1v6_090403.nc'
  Model docn missing file file1 = '/home/gregorylemieux/projects/cesm-inputdata/atm/cam/sst/sst_HadOIBl_bc_0.9x1.25_clim_c040926.nc'
  Model docn missing file domainfile = '/home/gregorylemieux/projects/cesm-inputdata/share/domains/domain.ocn.gx1v6.090206.nc'
Loading input file list: 'Buildconf/dice.input_data_list'
  Model dice missing file domain1 = '/home/gregorylemieux/projects/cesm-inputdata/ice/dice7/domain.ocn.x0.5.090227.nc'
  Model dice missing file file1 = '/home/gregorylemieux/projects/cesm-inputdata/ice/dice7/SSMI/ssmi_ifrac.clim.x0.5.090319.nc'
  Model dice missing file domainfile = '/home/gregorylemieux/projects/cesm-inputdata/share/domains/domain.ocn.gx1v6.090206.nc'
Client protocol gftp not enabled
Using protocol wget with user anonymous and passwd user@example.edu

and seems to hang. I think something is wrong with the connection still.

glemieux commented 5 years ago

Submitted a help ticket with the folks at UCAR CGD information systems group:http://www.cgd.ucar.edu/systems/documentation-toc/index.html. They found I had been blacklisted from the download site, probably for too many attempts. They've since removed that restriction, but that wasn't the primary cause for problems downloading the files.