feature/coupled-crow --- wave prep creates link to non-existent folder

[JessicaMeixner-NOAA]

On orion, several variables are defined with place_holder: Looking in the wave prep log all of the variables below have a placeholder definition: DMPDIR=/work/noaa/marine/place_holder RTMFIX=/work/noaa/marine/place_holder BASE_GIT=/work/noaa/marine/place_holder BASE_SVN=/work/noaa/marine/place_holder ---- do we need this variable anymore? HOMEobsproc_prep=/work/noaa/marine/place_holder/obsproc/obsproc_global.v3.2.3 HOMEobsproc_network=/work/noaa/marine/place_holder/obsproc/obsproc_global.v3.2.3 BASE_VERIF=/work/noaa/marine/place_holder/verif/global/tags/vsdb

And in jobs/JGLOBAL_WAVE_PREP: $NLN $DMPDIR/${WAVECUR_DID}.${RPDY} $ROTDIR/${WAVECUR_DID}.${RPDY}

creates a linked folder to the non-existent /work/noaa/marine/place_holder because of the DMPDIR definition. Additionally, we don't really need this chunk of code as currents are not received from RTOFS, so this could all be skipped for feature/coupled-crow anyways. Need to fix this.

[JessicaMeixner-NOAA]

I think this is at least in part related to definitions of variables not being defined on orion in workflow/platforms/orion.yaml

DMPDIR: "/work/noaa/marine/place_holder" NWPROD: "/work/noaa/marine/place_holder" RTMFIX: "/work/noaa/marine/place_holder" BASE_SVN: "/work/noaa/marine/place_holder" BASE_GIT: "/work/noaa/marine/place_holder"

For example in workflow/platforms/hera.yaml they are defined as follow: https://github.com/NOAA-EMC/global-workflow/blob/feature/coupled-crow/workflow/platforms/hera.yaml#L27-L30

@KateFriedman-NOAA is there a global dump location on orion I can set the default to for these variables?

I should probably also still add something to skip the part of the wave prep job above for the fully coupled model, but I also think the definition of the above variables is an issue as well.

[KateFriedman-NOAA]

@KateFriedman-NOAA is there a global dump location on orion I can set the default to for these variables?

@JessicaMeixner-NOAA Yes, you can set those variables as:

DMPDIR: "/work/noaa/global/glopara/dump"
NWPROD: "/work/noaa/global/glopara/nwpara"
RTMFIX: "/apps/contrib/NCEPLIBS/orion/fix/crtm_v2.3.0"
BASE_SVN: "/work/noaa/global/glopara/svn"
BASE_GIT: "/work/noaa/global/glopara/git"

RTMFIX will come from hpc-stack in the future but the fix files are on disk still.

FYI, most of those paths are set in the pre-CROW workflow in this part of setup_expt.py (for reference):

https://github.com/NOAA-EMC/global-workflow/blob/develop/ush/rocoto/setup_expt.py#L238

[JessicaMeixner-NOAA]

Thanks @KateFriedman-NOAA!!

It seems like BASE_SVN and BASE_GIT aren't really used but maybe that was again me not grepping w/out the case insensitive when I thought that.

[KateFriedman-NOAA]

@JessicaMeixner-NOAA BASE_GIT is used to define components like obsproc and fit2obs. See the use of BASE_GIT and BASE_SVN in the configs. BASE_SVN isn't used much anymore and can probably go now.

[WalterKolczynski-NOAA]

We should probably refactor to remove BASE_GIT as well. Pointing to an external catch-all directory like this isn't good practice. If obsproc and fit2obs need to remain external packages for some reason, there should be clearly named variables to point to them, rather than the nebulous "BASE_GIT".

[KateFriedman-NOAA]

@WalterKolczynski-NOAA I agree. We have far fewer external pieces coming from BASE_GIT now so we can change those (e.g. obsproc) to use a more full path without a variable. I'd also like to consider pulling things like fit2obs into the list of components that global-workflow checks out via checkout.sh/Exernals.cfg (and add to the build too). I'd like to consider that for obsproc but they don't yet provide build support for every machine (I've manually installed it in most R&D machines). The obsproc team is open to receiving build updates for the R&D machines though and once it uses hpc-stack that will be MUCH easier. Something to keep in mind for this year.

[JessicaMeixner-NOAA]

Fixed in PR#255