Theta fixes for LIGHT

[pwolfram]

This PR fixes errors arising from inaccurate subroutine arguments that were not an issue with other compilers. The theta-knl compiler was unable to run without these fixes.

[pwolfram]

Testing

Tested on -compset GMPAS-IAF -res T62_oEC60to30v3 to fix seg fault errors.

[mark-petersen]

@pwolfram Just FYI, our convention is that the first message on the PR is used as the merge message. Best to keep it short, and put testing documentation in a following comment.

From @pwolfram: Code was tested using a command of form

export ACME_VERSION=ACME_110417_debug

#export ACME_VERSION=ACME

export ACME_CASE=a03    

export NPROCS_OCN=1024

export DIR_ROOT=/home/pwolfram/Mark_comparison/

export CASE_ROOT=$DIR_ROOT/ACME_cases

export RUN_ROOT=/projects/OceanClimate/pwolfram

#********* step 1: set up directories

cd $DIR_ROOT

git clone git@github.com:ACME-Climate/ACME.git $ACME_VERSION

cd $ACME_VERSION

git submodule update --init

#********* step 2 & 3 create new case

export ACME_ROOT=/home/pwolfram/Mark_comparison/$ACME_VERSION

cd $ACME_ROOT/cime/scripts

./create_newcase -case $CASE_ROOT/$ACME_CASE -compiler intel -mach theta -project OceanClimate -compset GMPAS-IAF -res T62_oEC60to30v3

#******** step 4 setup

cd $CASE_ROOT/$ACME_CASE

export NPROCS_OTHERS=$(( NPROCS_OCN / 2 ))

export NPROCS_TOTAL=$(( NPROCS_OCN + NPROCS_OTHERS ))

./xmlchange -file env_mach_pes.xml  -id NTASKS_OCN  -val $NPROCS_OCN

./xmlchange -file env_mach_pes.xml  -id ROOTPE_OCN  -val $NPROCS_OTHERS

./xmlchange -file env_mach_pes.xml  -id NTASKS_ATM  -val $NPROCS_OTHERS

./xmlchange -file env_mach_pes.xml  -id NTASKS_CPL  -val $NPROCS_OTHERS

./xmlchange -file env_mach_pes.xml  -id NTASKS_WAV  -val $NPROCS_OTHERS

./xmlchange -file env_mach_pes.xml  -id NTASKS_GLC  -val $NPROCS_OTHERS

./xmlchange -file env_mach_pes.xml  -id NTASKS_ICE  -val $NPROCS_OTHERS

./xmlchange -file env_mach_pes.xml  -id NTASKS_ROF  -val $NPROCS_OTHERS

./xmlchange -file env_mach_pes.xml  -id NTASKS_LND  -val $NPROCS_OTHERS

./xmlchange -file env_batch.xml -subgroup case.run -id JOB_QUEUE -val default

./xmlchange -file env_batch.xml -subgroup case.run -id JOB_WALLCLOCK_TIME -val "01:00:00"

# set up particles (assumes file already built)

# cp ~/LIGHT_optimization/ACME_cases/20171026.smoketest.particles.GMPAS-IAF.T62_oEC60to30v3.theta/user_nl_mpaso .

# vimdiff /projects/OceanClimate/pwolfram//20171026.smoketest.particles.GMPAS-IAF.T62_oEC60to30v3.theta/run/streams.ocean SourceMods/src.mpaso/streams.ocean 

export ACME_CASE_PARTICLES=$CASE_ROOT/a02

cd $CASE_ROOT/$ACME_CASE

cp $ACME_CASE_PARTICLES/user_nl_mpaso user_nl_mpaso

cp $ACME_CASE_PARTICLES/SourceMods/src.mpaso/streams.ocean SourceMods/src.mpaso/streams.ocean

# to set debug to be true

./xmlchange -file env_build.xml -id DEBUG -val TRUE

cd $CASE_ROOT/$ACME_CASE

./case.setup

./case.build

./case.submit

[pwolfram]

Thanks @mark-petersen, so to be a pain by breaking convention. Thanks for revising the description for me! Your help was invaluable in this fix-- it is much appreciated!!!

[mark-petersen]

Tested with ziso/20km/default, which runs LIGHT, with gnu/optimized and intel/debug on grizzly.

[pwolfram]

@mark-petersen, agreed. The take-away here is that for whatever reason the Intel-KNL compiler toolchain is more strict with respect to Fortran standards than just GNU or Intel alone. It was a good chance to clean up some of the LIGHT debugging anyway so we will be stronger at the end of this process regardless.

[mark-petersen]

Tested merged version, bfb with previous on nightly test suite with both intel and gnu.