Add bash linting to CI. Cleanup .sh scripts a bit. Address .sh bugs. Adds -v Verbose option. (#2218) Remove nowarn Intel compiler flag

[BrianCurtis-NOAA]

Commit Queue Requirements:

• [x] Fill out all sections of this template.
• [x] All sub component pull requests have been reviewed by their code managers.
• [x] Run the full Intel+GNU RT suite (compared to current baselines) on either Hera/Derecho/Hercules

• [x] Commit 'test_changes.list' from previous step

  Description:

• Add bash linting to CI:
  • uses superlinter to check for consistent bash code writing
• Cleans up .sh scripts to comply with superlinter
• Cleans up .sh scripts to be more consistent, easier to read.
• Add's -v verbose option if debugging outputs needed, otherwise simplifies rt.sh run echo's.
• Addresses smaller bugs
  • quota/timeout search logic adjusted.
  • check for dirs existing (DISKNM, STMP, PTMP) before starting.
  • adjustments/cleanup to ecflow/rocoto sections
  • rt.sh will attempt to start ecflow, and only stop ecflow if it started from rt.sh.
  • fix for issue where run_dir will not delete properly.
• Remove -nowarn compiler flag from cmake/Intel.cmake
• Explicitly disable two warnings:
  • warning #5462: Global name too long.
  • remark #7712: This variable has not been used.

    Commit Message:

• UFSWM
• Add bash linting to CI:
  • uses superlinter to check for consistent bash code writing
• Cleans up .sh scripts to comply with superlinter
• Cleans up .sh scripts to be more consistent, easier to read.
• Add's -v verbose option if debugging outputs needed, otherwise simplifies rt.sh run echo's.
• Addresses smaller bugs
  • quota/timeout search logic adjusted.
  • check for dirs existing (DISKNM, STMP, PTMP) before starting.
  • adjustments/cleanup to ecflow/rocoto sections
  • rt.sh will attempt to start ecflow, and only stop ecflow if it started from rt.sh.
  • fix for issue where run_dir will not delete properly.
• FV3: Address compiler warnings
  • atmos_cubed_sphere: Address compiler warnings.

Priority:

• Normal

Git Tracking

UFSWM:

• Closes #2109
• Closes #1984
• Closes #2087
• Closes #2148

  Sub component Pull Requests:

  • None

  ---

  Changes

  Regression Test Changes (Please commit test_changes.list):

  • No Baseline Changes.

  Input data Changes:

  • None.

  Library Changes/Upgrades:

  • No Updates

  ---

  Testing Log:

  • RDHPCS
    • [x] Hera
    • [x] Orion
    • [x] Hercules
    • [x] Jet
    • [x] Gaea
    • [x] Derecho
  • WCOSS2
    • [x] Dogwood/Cactus
    • [ ] Acorn
  • [x] CI
  • [ ] opnReqTest (complete task if unnecessary)

[BrianCurtis-NOAA]

I've brought over all info from #2218 and am running the full RT suite on Hera to double check no baselines are changing.

[zach1221]

@BrianCurtis-NOAA hercules, and orion are running fine. Derecho keeps aborting though when I try to use ecflow. If I attempt to run sequentially, without a workflow, rt.sh will just exit. Can you try running a job on Derecho as well, to see the behavior?

[zach1221]

Also, ORT fails on control_p8_gnu.

• echo 'rt_utils.sh: std_base: Computing PET bounds and tasks.' rt_utils.sh: line 83: RTVERBOSE: unbound variable Died with error code 1

[BrianCurtis-NOAA]

@zach1221 try adding to the opnreq script export RTVERBOSE=false early enough that when you source rt_utils it will see RTVERBOSE.

I'll run on Derecho this AM and see what I can see.

[zach1221]

Still working the derecho issue, fyi. Everything else should be done.

[zach1221]

Ok jobs are running now on Derecho.

[zach1221]

Ok we're ready to merge finally.

[BrianCurtis-NOAA]

Almost, I would like a little time to see if i can get Acorn finished.

[BrianCurtis-NOAA]

Skipping Acorn. Start merge.

[zach1221]

@BrianCurtis-NOAA fv3atm merged. Hash: https://github.com/NOAA-EMC/fv3atm/commit/979bcab28f63b37411698cd9d23d04d0b0fe3a7e

[BrianCurtis-NOAA]

I have the FV3 updated, but waiting on stochastic_physics before pushing. Who normally merges in stochastic_physics?

[zach1221]

I have the FV3 updated, but waiting on stochastic_physics before pushing. Who normally merges in stochastic_physics?

Oh, Phil Pegion generally. I can leave a comment there.

[zach1221]

Ok, @BrianCurtis-NOAA . We should be ready now, Stoch Physics is merged.

[DeniseWorthen]

@BrianCurtis-NOAA Shouldn't you link other bug reports so that they'll be closed ? 2087 maybe 2150 and also 2148?

[DeniseWorthen]

I've added the additional issues close. I'm not sure if #2150 is fixed, so I'll leave that open for now.