List of supported platforms for the release

[climbfuji]

Here is a link to a spreadsheet that lists the supported platforms/compilers and who has access to these:

https://docs.google.com/spreadsheets/d/122uasMhD8aF_s6jUpy7t_Io5JSNGYOmBddtT6kh0dSQ/edit#gid=0

Please use this spreadsheet to indicate the readiness for testing and who is testing on which platform (maybe also indicate success/failures - this has not been set up yet in the spreadsheet). We can also use this GitHub issue to report successes/failures.

[arunchawla-NOAA]

The platforms status spreadsheet has been rejigged to reflect the plans.

[arunchawla-NOAA]

@ceceliadid I am updating the wiki documentation for the supported platforms and the documentation links for the RDHPCS platforms require access that people may not have

[arunchawla-NOAA]

@climbfuji @pjpegion @mark-a-potts

Can you review this wiki

https://github.com/ufs-community/ufs/wiki/Supported-Platforms-and-Compilers

and let me know if we can close this ticket? Do we want to add names for Tier 3 / Tier 4 platforms or leave it out for now

[ceceliadid]

Before closing please add information on what people are expected to do on configurable platforms. Is there a blessed user directory installation of nceplibs/nceplibs-external that they're supposed to use? Hopefully they're not expected to install libs themselves on Stampede2 and orion.

[jedwards4b]

On all tier 1 and 2 systems support libraries should be installed and integrated with cime. Users don't really need to know the details of where the are.

On Sat, Feb 22, 2020, 11:30 ceceliadid notifications@github.com wrote:

Before closing please add information on what people are expected to do on configurable platforms. Is there a blessed user directory installation of nceplibs/nceplibs-external that they're supposed to use? Hopefully they're not expected to install libs themselves on Stampede2 and orion.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ufs-community/ufs-mrweather-app/issues/67?email_source=notifications&email_token=ABOXUGBA3MI2DQKTFEF4EK3REFVNDA5CNFSM4KKZRJV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMVHJMY#issuecomment-589984947, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABOXUGE4HJ63BUOSW74TFODREFVNDANCNFSM4KKZRJVQ .

[arunchawla-NOAA]

Tier 1 yes, but Tier 2 the issue is we do not have a central place to put the libraries. But CIME testing is integrated and it should work right out of the box (with the exception of having to identify where the libraries are).

Nevertheless it will be worthwhile for some independent testers to try and install the libraries as well as CIME. We are already seeing that with the test teams and this can be expanded further.

Instructions are on the wiki pages as well as CIME guide. We should see how easy it is to get to them.

[ceceliadid]

Once again, we have UFS applications that are not going to be using CIME and need to use installations of nceplibs and nceplibs-external on Stampede2 and Orion. We need to be clear about what to do in that case. I need this info to run the WM GST on Stampede2.

[jedwards4b]

At least on stampede the nceplibs (although in a user directory) should be available for all to use.

On Sat, Feb 22, 2020 at 2:24 PM ceceliadid notifications@github.com wrote:

Once again, we have UFS applications that are not going to be using CIME and need to use installations of nceplibs and nceplibs-external on Stampede2 and Orion. We need to be clear about what to do in that case.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ufs-community/ufs-mrweather-app/issues/67?email_source=notifications&email_token=ABOXUGBKO7HY2C4T4IXEVCLREGJXTA5CNFSM4KKZRJV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMVK4OA#issuecomment-589999672, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABOXUGGL4OQ4DVK7FBLSP53REGJXTANCNFSM4KKZRJVQ .

-- Jim Edwards

CESM Software Engineer National Center for Atmospheric Research Boulder, CO

[arunchawla-NOAA]

Cecilia

The NCEPLIBS external and NCEPLIBS wiki have step by step installation instructions. The model build has instructions on what paths need to be set. Are we missing something ?

Sent from my iPhone

On Feb 22, 2020, at 4:47 PM, jedwards4b notifications@github.com wrote:

At least on stampede the nceplibs (although in a user directory) should be available for all to use.

On Sat, Feb 22, 2020 at 2:24 PM ceceliadid notifications@github.com wrote:

Once again, we have UFS applications that are not going to be using CIME and need to use installations of nceplibs and nceplibs-external on Stampede2 and Orion. We need to be clear about what to do in that case.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ufs-community/ufs-mrweather-app/issues/67?email_source=notifications&email_token=ABOXUGBKO7HY2C4T4IXEVCLREGJXTA5CNFSM4KKZRJV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMVK4OA#issuecomment-589999672, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABOXUGGL4OQ4DVK7FBLSP53REGJXTANCNFSM4KKZRJVQ .

-- Jim Edwards

CESM Software Engineer National Center for Atmospheric Research Boulder, CO — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub, or unsubscribe.

[ceceliadid]

Thanks @jedwards4b I know that's what you said on a recent call but I think Linlin got confused when she was trying to run the WM GST on Stampede2 and didn't know if she needed to build the libraries. It may just be a documentation thing. So what the WM UG says is that:

"You can either manually set those environment variables in your shell, or if you are on one of the supported platforms (Tier 1) the easiest way of setting those variables is by using environment module and load all required modules. Modulefiles for all supported platforms are located in modulefiles//fv3."

This is the example given on hera: $ cd modulefiles/hera.intel $ module use $(pwd) $ module load fv3 $ cd ../..

Will that work to get the modulefiles on Stampede2? Do you think it will also work for Orion?

I put in a ticket to address the Tier 1 terminology vs the new pre-configured/configurable terminology in the UG documentation - that may have been confusing.

[ceceliadid]

@arunchawla-NOAA I don't think we want people especially GST folks to have to go through building nceplibs-external/nceplibs on Stampede2 and Orion if they don't have to. We should check that loading modules works as Jim and the WM UG says it will on these "configurable" platforms.

[arunchawla-NOAA]

Orion is still an unstable platform. I would keep it at level 2, I. e. Libraries are not built and available. Technically the tier 1 platforms are Cheyenne and the NOAA RDHPCS machines

Sent from my iPhone

On Feb 22, 2020, at 5:25 PM, ceceliadid notifications@github.com wrote:

 @arunchawla-NOAA I don't think we want people especially GST folks to have to go through building nceplibs-external/nceplibs on Stampede2 and Orion if they don't have to. We should check that loading modules works as Jim and the WM UG says it will on these "configurable" platforms.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[ceceliadid]

@arunchawla-NOAA I don't care as much about Orion right now but would like to get Stampede2 sorted out. That platform is level 2 by definition since its libs are not in a central location. But we still need to know what users are supposed to do on it. I would think once Orion is stable it can follow the precedent set by Stampede2 in terms of module loading.

[arunchawla-NOAA]

Module loading would only be for tier 1 platforms. For now they can point to already existing libraries on stampede, right ?

Sent from my iPhone

On Feb 22, 2020, at 5:35 PM, ceceliadid notifications@github.com wrote:

 @arunchawla-NOAA I don't care as much about Orion right now but would like to get Stampede2 sorted out. That platform is level 2 by definition since its libs are not in a central location. But we still need to know what users are supposed to do on it. I would think once Orion is stable it can follow the precedent set by Stampede2 in terms of module loading.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[jedwards4b]

@ceceliadid I think that you need to make it clear that when using cime on stampede you don't need to load any modules or environment variables - they are defined in cimes config_machine.xml file for that platform.

[ceceliadid]

@jedwards4b I am trying to get instructions on how to access the nceplibs-external and nceplibs installations on Stampede2 for folks who are NOT using cime. The instructions above are from the WM UG section Compiling the Code without an Application.

[arunchawla-NOAA]

Cecelia

On stampede2 the libraries are located here

/work/01118/tg803972/stampede2/UFS/NCEP_LIBS_ALL.1.0.0.beta01

If you set the relevant paths in build script then they should work. This is not a permanent location but should work. You should check that everything is there

Arun Chawla Chief Engineering & Implementation Branch Room 2083 National Center for Weather & Climate Prediction 5830 University Research Court College Park, MD 20740 Phone : 301-683-3740 Mobile : 240-564-5675 Fax : 301-683-3703

On Sat, Feb 22, 2020 at 5:50 PM ceceliadid notifications@github.com wrote:

@jedwards4b https://github.com/jedwards4b I am trying to get instructions on how to access the nceplibs-external and nceplibs installations on Stampede2 for folks who are NOT using cime.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ufs-community/ufs-mrweather-app/issues/67?email_source=notifications&email_token=AL5NYI3KVOZ3DSD6CDMVRNTREGT35A5CNFSM4KKZRJV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMVMNYA#issuecomment-590005984, or unsubscribe https://github.com/notifications/unsubscribe-auth/AL5NYI6QF2SOU542NKBGWP3REGT35ANCNFSM4KKZRJVQ .

[ceceliadid]

Thanks @arunchawla-NOAA I think the WM UG needs to be a lot clearer about this than it currently is about where to find these libraries and how to set the path for the platform - I don't know how anyone would figure this out on their own for Stampede2. So far I haven't been editing the WM UG - @llpcarson would you be the person taking care of this?

So basically the user needs to go through all the internal and third party libraries listed on this page which is also what is in build.sh: https://ufs-mr-weather-app.readthedocs.io/projects/ufs-weather-model/en/ufs-v1.0.0.beta01doc/CompilingCodeWithoutApp.html

and set every path to the one you just provided and that should work?

[ceceliadid]

@arunchawla-NOAA I guess the question is if the libs are all at the top dir of the path, so you don't have to reference any subdirectories or anything, and do something like: /work/01118/tg803972/stampede2/UFS/NCEP_LIBS_ALL.1.0.0.beta01/ESMF/

[arunchawla-NOAA]

For the folks not using CIME environment variables have to be set in the build. These variables do not assume any layout on where the libraries are built and kept (that is useful to have as in some places they may not be in a vertical layout). When the developers install libraries then a shell script is created that they can then source to set the paths and run the build. For preinstalled libraries right now they have to manually set the path, we will review the module files in the coming days as the libraries location is centralized.

For CIME the path has to be set in an xml config file. For pre configured platforms this will already be there (may have to be updated once the final locations with no beta tags are placed). For configurable platforms Jim and Rocky can say how well that is documented.

Arun Chawla Chief Engineering & Implementation Branch Room 2083 National Center for Weather & Climate Prediction 5830 University Research Court College Park, MD 20740 Phone : 301-683-3740 Mobile : 240-564-5675 Fax : 301-683-3703

On Sat, Feb 22, 2020 at 7:29 PM ceceliadid notifications@github.com wrote:

@arunchawla-NOAA https://github.com/arunchawla-NOAA I guess the question is if the libs are all at the top dir of the path, so you don't have to reference any subdirectories or anything, and do something like: /work/01118/tg803972/stampede2/UFS/NCEP_LIBS_ALL.1.0.0.beta01/ESMF/

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ufs-community/ufs-mrweather-app/issues/67?email_source=notifications&email_token=AL5NYI7XEKUP3MFDUW2YST3REG7ONA5CNFSM4KKZRJV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMVODRA#issuecomment-590012868, or unsubscribe https://github.com/notifications/unsubscribe-auth/AL5NYI5QQZS2HO5ZFJBJENDREG7ONANCNFSM4KKZRJVQ .

[ceceliadid]

Thanks @arunchawla-NOAA The question is whether below is correct for the build.sh on stampede2 when people are not using CIME.

export NEMSIO_INC=/work/01118/tg803972/stampede2/UFS/NCEP_LIBS_ALL.1.0.0.beta01

export NEMSIO_LIB=/work/01118/tg803972/stampede2/UFS/NCEP_LIBS_ALL.1.0.0.beta01/libnemsio.a

export BACIO_LIB4=/work/01118/tg803972/stampede2/UFS/NCEP_LIBS_ALL.1.0.0.beta01/libbacio.a

export SP_LIBd=/work/01118/tg803972/stampede2/UFS/NCEP_LIBS_ALL.1.0.0.beta01/libsp_d.a

export W3EMC_LIBd=/work/01118/tg803972/stampede2/UFS/NCEP_LIBS_ALL.1.0.0.beta01/libw3emc_d.a

export W3NCO_LIBd=/work/01118/tg803972/stampede2/UFS/NCEP_LIBS_ALL.1.0.0.beta01/libw3nco_d.a

export NETCDF=/work/01118/tg803972/stampede2/UFS/NCEP_LIBS_ALL.1.0.0.beta01

export ESMFMKFILE=/work/01118/tg803972/stampede2/UFS/NCEP_LIBS_ALL.1.0.0.beta01/esmf.mk

[climbfuji]

Support levels are getting mixed up here, and this causes confusion. Per definition, level 2 support = configurable platform means everything is tested and known to work, including the entire workflow.

Users have to install NCEPLIBS (and if necessary also parts of NCEPLIBS-external) by themselves, unless the stubborn HPC admins agree to create a shared common, user-independent space and elevate it to a level 1 platform.

If it stays level 2, we should provide step-by-step instructions as we do for macOS, Linux, and all the other platforms, and CIME has to look for the relevant environment variables being set - no hardcoded locations. This is what was agreed upon for the release.

Whether you want to do this for the intermittent GST is a different story, but I would consider it beneficial to make building the libraries part of the GST - we need external users to test it.

[climbfuji]

On all tier 1 and 2 systems support libraries should be installed and integrated with cime. Users don't really need to know the details of where the are. …

No! On level 2 platforms users need to install the library by themselves. CIME will look for the relevant environment variables like for macOS. Nothing else.

[arunchawla-NOAA]

I agree with Dom. Support level 2 was decided to be where people have to install their own set of libraries. This has been discussed let us not keep bringing this back.

Right now the level 1 platforms are Cheyenne, Gaea, Hera and Jet. This is where the final release tag libraries will be installed and there will be module files for them.

There is no bandwidth to do more given the timeline, so please do not add more systems as support level 1.

Sent from my iPhone

On Feb 24, 2020, at 9:34 AM, Dom Heinzeller notifications@github.com wrote:

 On all tier 1 and 2 systems support libraries should be installed and integrated with cime. Users don't really need to know the details of where the are. …

No! On level 2 platforms users need to install the library by themselves. CIME will look for the relevant environment variables like for macOS. Nothing else.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[climbfuji]

Thanks, Arun. I can create the step-by-step instructions for stampede and orion, I have accounts on both systems. For stampede I'd need to know which modules were used until now (compiler version, MPI version, netCDF version). Thanks very much in advance!

[jedwards4b]

Okay. I guess I didn't have a clear understanding of what you meant by level 2. I will update Stampede to expect users to set environment variables. The currently used modules on stampede are here: https://github.com/ESMCI/cime/blob/ufs_release_v1.0/config/ufs/machines/config_machines.xml#L348

[climbfuji]

Thanks Jim, I will use those with the upcoming beta03 tag of NCEPLIBS on stampede.

[jedwards4b]

Users are now required to set the following environment variables on stamepede before they can run cime: DIN_LOC_ROOT - location of input data NCEPLIBS_DIR - location of installed NCEPLIBS ESMFMKFILE - location of esmf.mk

[ceceliadid]

@arunchawla-NOAA @climbfuji @jedwards4b Wow, it was not my understanding at all that all the graduate students using the allocation we got on Stampede for the GST are going to have to install the libraries themselves, rather than using an installation of the libs that will work. Stampede2 was originally supposed to be a fully supported platform and when it wsa demoted I was assured users would still not have to build libs.

[ceceliadid]

@climbfuji I can still ask some GST folks to build the libs, I just don't want it to be the default for them.

[climbfuji]

The GST as a temporary step is not the problem (for me at least). But the UFS release when it goes out the door on March 6 must require users to build the libraries by themselves on configured but not supported platforms.

[ceceliadid]

This is the current definition of level 2 support: Configurable platforms are platforms where all of the required libraries for building community releases of UFS models and applications are expected to install successfully, but are not available in a central place (it may be a user directory). Applications and models are expected to build and run once the required libraries are built or located.

This implies that it is permissible to have a non-centralized location for the libs. Do you want to change the wording and say they always have to be built?

[jedwards4b]

@ceceliadid how about if we document the environment variables that need to be set and provide suggested values for them (that should work) with some wording about these not being official install paths? That way the user doesn't need to build these libraries or download the inputdata unless they choose to do so.

[ceceliadid]

@jedwards4b That seems okay to me, but it seems like a more general policy question that others will have to answer.

I tried building the WM on Stampede2 without CIME with the paths that Arun provided (from a few comments back) and it bails out quickly with CMake Error at cmake/FindESMF.cmake:12 (file): file STRINGS file "/work/01118/tg803972/stampede2/UFS/NCEP_LIBS_ALL.1.0.0.beta01/esmf.mk" cannot be read

Is that a permission problem with the lib dir path? I can see the directory work/01118/tg803972/stampede2 but I can't see anything in the UFS dir.

[jedwards4b]

@ceceliadid ESMFMKFILE should be set to /work/01118/tg803972/stampede2/UFS/NCEPLIBS-external/build/install/lib64/esmf.mk

[ceceliadid]

@jedwards4b Thanks but that still gives me: ESMFMKFILE: /work/01118/tg803972/stampede2/UFS/NCEPLIBS-external/build/install/lib64/esmf.mk CMake Error at cmake/FindESMF.cmake:12 (file): file STRINGS file "/work/01118/tg803972/stampede2/UFS/NCEPLIBS-external/build/install/lib64/esmf.mk" cannot be read.

[climbfuji]

Same problem here. This is why I am insisting on project spaces and not user directories. Instructions for compiling the libraries on stampede will be available shortly, straightforward and quick.

login2(422)$ ls /work/01118/tg803972/stampede2/UFS/NCEPLIBS-external/build/install/lib64/esmf.mk
ls: cannot access /work/01118/tg803972/stampede2/UFS/NCEPLIBS-external/build/install/lib64/esmf.mk: Permission denied
login2(423)$ whoami
tg854455
login2(424)$ groups
G-81589

[uturuncoglu]

I am not sure way you can't access because Jim is not in my group and he could access. Anyway, you could also install externals also using following information.

# Stampede2
source /opt/apps/lmod/lmod/init/sh
module purge
module load TACC python/2.7.15 intel/18.0.2 cmake/3.10.2
module rm mvapich2
module load impi/18.0.2 pnetcdf/1.11.0
module load netcdf/4.6.2
module load cmake
module load hdf5/1.10.4

# NCEPLIBS-external
git clone --recursive -b master https://github.com/NOAA-EMC/NCEPLIBS-external.git
cd NCEPLIBS-external/
mkdir build-all
cd build-all
NETCDF=$TACC_NETCDF_DIR CC=mpicc FC=mpif90 CXX=mpicxx cmake -DBUILD_MPI=OFF -DBUILD_NETCDF=OFF -DBUILD_PNG=OFF -DBUILD_JASPER=OFF -DMPITYPE=intelmpi -DCMAKE_INSTALL_PREFIX=$PWD/install ..
make -j 4

# NCEPLIBS
git clone --recursive -b ufs-v1.0.0.beta01 https://github.com/NOAA-EMC/NCEPLIBS.git NCEP_LIBS_ALL.1.0.0.beta01
cd NCEP_LIBS_ALL.1.0.0.beta01
mkdir build-all
cd build-all
NETCDF=$TACC_NETCDF_DIR HDF5=$TACC_HDF5_DIR CC=mpicc FC=mpif90 CXX=mpicxx cmake -DMPITYPE=intelmpi -DCMAKE_INSTALL_PREFIX=$PWD/install -DEXTERNAL_LIBS_DIR=/work/01118/tg803972/stampede2/UFS/NCEPLIBS-external/build/install ..
make

[jedwards4b]

@ceceliadid Sorry - those directories and files are world readable, I don't understand why you can 't read them. But I guess we've validated Dom's viewpoint.

[climbfuji]

What the sysadmins did on gaea a while ago is that they made the very top-level directories readable to the respective groups only. Nobody realized that. If that's not the case, maybe they put ACL restrictions on one of the directories in the path?

[climbfuji]

See here for the PR containing the setup instructions for Stampede with Intel: https://github.com/NOAA-EMC/NCEPLIBS-external/pull/29

[ceceliadid]

I can see all the dirs in work/01118/tg803972/stampede2 just not read into any of them. Anyway @climbfuji I guess I'm coming around to needing to build libs for now. Thanks for the instructions I'll give it a shot.

[climbfuji]

Follow-up question: does this also affect the files in DIN_LOC_ROOT? On the generic platforms, the data is downloaded using the ./check_input_data --download command in lack of a shared DIN_LOC_ROOT location.

[jedwards4b]

I have no way of knowing - can you try and let me know? DIN_LOC_ROOT=/work/01118/tg803972/stampede2/UFS/ufs_inputdata

On Mon, Feb 24, 2020 at 4:27 PM Dom Heinzeller notifications@github.com wrote:

Follow-up question: does this also affect the files in DIN_LOC_ROOT? On the generic platforms, the data is downloaded using the ./check_input_data --download command in lack of a shared DIN_LOC_ROOT location.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ufs-community/ufs-mrweather-app/issues/67?email_source=notifications&email_token=ABOXUGAXDJZ3YLJYYWJ56QDRERJU5A5CNFSM4KKZRJV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMZ5U7I#issuecomment-590600829, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABOXUGEXKXMYHMFRCZUNGDDRERJU5ANCNFSM4KKZRJVQ .

-- Jim Edwards

CESM Software Engineer National Center for Atmospheric Research Boulder, CO

[climbfuji]

login2(426)$ ls /work/01118/tg803972/stampede2/UFS/ufs_inputdata
ls: cannot access /work/01118/tg803972/stampede2/UFS/ufs_inputdata: Permission denied

so I fear we need to use the download script there as well ... or get it from HPSS / Niagara?

[climbfuji]

Ok, so right now this is not working on orion, because it wants me to set DIN_LOC_ROOT. I assume the "transition" to using the download script is not complete (also, the hashes in manage externals have not been updated to include the generic linux pieces for the two cime repositories - sidenote).

login2(495)$  ./cime/scripts/create_newcase --case c96-gfsv15p2-gnu --compset GFSv15p2 --res C96 --workflow ufs-mrweather --machine stampede2-skx
Compset longname is FCST_ufsatm%v15p2_SLND_SICE_SOCN_SROF_SGLC_SWAV
Compset specification file is /scratch/06146/tg854455/ufs-mrweather-app/my-ufs-sandbox/src/model/FV3/cime/cime_config/config_compsets.xml
Automatically adding SESP to compset
Compset forcing is
ATM component is UFSATM Atmosphere with:CCPP physics version 15p2
LND component is Stub land component
ICE component is Stub ice component
OCN component is Stub ocn component
ROF component is Stub river component
GLC component is Stub glacier (land ice) component
WAV component is Stub wave component
ESP component is Stub external system processing (ESP) component
Pes     specification file is /scratch/06146/tg854455/ufs-mrweather-app/my-ufs-sandbox/src/model/FV3/cime/cime_config/config_pes.xml
Compset specific settings: name is RUN_STARTDATE and value is 2019-08-29
Compset specific settings: name is START_TOD and value is 0
Compset specific settings: name is COMP_CLASSES and value is ATM
Compset specific settings: name is CHECK_TIMING and value is FALSE
Machine is stampede2-skx
Pes setting: grid match    is a%C96
Pes setting: compset_match is ufsatm
Pes setting: grid          is a%C96_l%null_oi%null_r%null_g%null_w%null_z%null_m%null
Pes setting: compset       is FCST_ufsatm%v15p2_SLND_SICE_SOCN_SROF_SGLC_SWAV_SESP
Pes setting: tasks       is {'NTASKS_ATM': 108}
Pes setting: threads     is {'NTHRDS_ATM': 1}
Pes setting: rootpe      is {}
Pes setting: pstrid      is {}
Pes other settings: {}
Pes comments: none
 Compset is: FCST_ufsatm%v15p2_SLND_SICE_SOCN_SROF_SGLC_SWAV_SESP
 Grid is: a%C96_l%null_oi%null_r%null_g%null_w%null_z%null_m%null
 Components in compset are: ['ufsatm', 'slnd', 'sice', 'socn', 'srof', 'sglc', 'swav', 'sesp']
Using project from config_machines.xml: TG-ATM190009
No charge_account info available, using value from PROJECT
ufs model version found: 450047f
Batch_system_type is slurm
job is case.chgres USER_REQUESTED_WALLTIME None USER_REQUESTED_QUEUE None
job is case.run USER_REQUESTED_WALLTIME None USER_REQUESTED_QUEUE None
job is case.gfs_post USER_REQUESTED_WALLTIME None USER_REQUESTED_QUEUE None
job is case.st_archive USER_REQUESTED_WALLTIME None USER_REQUESTED_QUEUE None
 Creating Case directory /scratch/06146/tg854455/ufs-mrweather-app/my-ufs-sandbox/c96-gfsv15p2-gnu
login2(496)$ cd c96-gfsv15p2-gnu
login2(497)$ ./case.setup
ERROR: Undefined env var 'DIN_LOC_ROOT'

[jedwards4b]

@climbfuji DIN_LOC_ROOT is where the data is OR where the download will put the data if it isn't already there, there is no separate script.

[climbfuji]

Sure, but I don't need to set this on macos or generic linux. I assume that all supported but not preconfigured platforms would be treated the same way? Only preconfigured platforms would have the data in a central location, predefined as DIN_LOC_ROOT?

[jedwards4b]

The issue is that we have some inconsistency - on macos and linux we are defining <DIN_LOC_ROOT>$ENV{UFS_INPUT}/ufs-inputdata</DIN_LOC_ROOT>

So we should probably define it the same way on stampede and Orian

[climbfuji]

Yes, I think that would be great. Thanks for adjusting the stampede configuration so quickly yesterday. I am currently working on orion. Tomorrow I'll do gaea (down for maintenance today).