search

ESMCI / cime

Common Infrastructure for Modeling the Earth
http://esmci.github.io/cime
Other
161 stars 206 forks source link

Remove DIN_LOC_ROOT_CLMFORC variable in favor of softlinks on machines that need it #3097

Open ekluzek opened 5 years ago

ekluzek commented 5 years ago

In our CSEG meeting this morning we decided we should remove the DIN_LOC_ROOT_CLMFORC env variable, who's purpose is to put DATM forcing files on a different disk than the main DIN_LOC_ROOT directory. The current check_input_data script can't figure out this distinction so it fails when trying to download data. The data is stored in such a way to assume DIN_LOC_ROOT_CLMFORC=$DIN_LOC_ROOT/atm/datm7, and if it's different than that it won't work. Some discussion on this is here...

https://bb.cgd.ucar.edu/clm5-atmospheric-forcings-data

On machines that need it we can use softlinks. So we'll start doing that on cheyenne for example.

ekluzek commented 5 years ago

Looking at config_machines.xml for cesm I see:

These set it and shouldn't...

centos7-linux homebrew

This is wrong, but probably doesn't matter...

gaea

This definitively looks wrong to me (these wrongly copied how it was setup from cheyenne rather than the others):

lonestar5 stampede2-skx stampede2-knl

The ones that do use it:

cheyenne hobart izumi laramie

ekluzek commented 5 years ago

Looks like there are four machines for e3sm that set it and shouldn't.

jedwards4b commented 5 years ago

Can your PR please include fixes for the cesm config_machines.xml?

ekluzek commented 5 years ago

@jedwards4b I'm thinking this should be done in two steps. We should setup the softlinks and then change it so that all machines use this...

DIN_LOC_ROOT_CLMFORC=$DIN_LOC_ROOT/atm/datm7

The machines I don't have access to though are:

lonestar stampede gaea (although I suspect this doesn't have the data there so doesn't matter) two of the e3sm machines

So who creates the softlinks for those machines?

ekluzek commented 5 years ago

Oh, I missed the next step. The first step is to do the softlinks and make DIN_LOC_ROOT_CLMFORC be consistent everywhere. That would be the first PR. Then the next step would be to remove DIN_LOC_ROOT_CLMFORC everywhere.

billsacks commented 2 years ago

@ekluzek is this still wanted? If so, we think it belongs in https://github.com/ESMCI/ccs_config_cesm now.

briandobbins commented 2 years ago

Just adding a note (because I didn't understand how this worked) - by changing this on OTHER systems external to NCAR to use $DIN_LOC_ROOT/atm/datm7, they're able to download the data fine. This is being tested right now on the cloud system, and I'll update the containers later. I think because people use "Cheyenne" as a base machine entry to modify for new systems, they might not realize this.

I'm sure I'm missing some complexity, but Erik solved in 15 minutes what would've been a messy, long work-around for me.