CASE/feature/p7.1 doesn't work for C384L64

lipan-NOAA commented 2 years ago

Case Details:

@bbakernoaa @zhanglikate @rmontuoro

The global workflow is feature/p7.1; ufs_coupled.fd is feature/p7.1.

expdir is at /scratch2/NCEPDEV/stmp1/Li.Pan/chem202107/firex;

the forecasting job failed : application called MPI_Abort(comm=0x84000002, 1) - process 235 application called MPI_Abort(comm=0x84000004, 1) - process 0 slurmstepd: error: STEP 21878373.0 ON h18c26 CANCELLED AT 2021-08-25T01:29:02 srun: Job step aborted: Waiting up to 32 seconds for job step to finish.

Model Configuration Details

Resolution: C384L64
Begin Date: 2019070100
End Date:2019070200
Cold Start: True or False
Forecast length (FHMAX_GFS): 24 hours
FV3 output: every 6 hours

Aerosol Model Configuration

HOMEgfs=/scratch2/NCEPDEV/naqfc/Li.Pan/save/aerosol-workflow expdir=/scratch2/NCEPDEV/stmp1/Li.Pan/chem202107/firex logdir=/scratch2/NCEPDEV/stmp1/Li.Pan/COM/firex/logs/2019070100/gfs.forecast.highres.log rcfile=/scratch2/NCEPDEV/naqfc/Li.Pan/save/data/NASA/rc.firex

Measurements/Models available for comparison

zhanglikate commented 2 years ago

@lipan-NOAA The workflow of P7.1 is not working as I reported before last week. So you'd better switch to the older workflow. The older workflow is working with P7.1 as what I did, and you only need to update one place in the namelist as I did.

lipan-NOAA commented 2 years ago

I did it as you suggested, the only difference is resolution but it didn’t work either. The errors are identical.

Li

On Tue, Aug 24, 2021 at 10:01 PM Kate.Zhang-NOAA @.***> wrote:

@lipan-NOAA https://github.com/lipan-NOAA The workflow of P7.1 is not working as I reported before last week. So you'd better switch to the older workflow. The older workflow is working with P7.1 as what I did, and you only need to update one place in the namelist as I did.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/noaa-oar-arl/UFS-Aerosol-Config/issues/21#issuecomment-905109123, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGJX5OSTD5XDNT3ZXEE2EC3T6RFITANCNFSM5CYAKLEA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

-- Li Pan IMSG at NOAA/NCEP/EMC NCWCP RM #2771 5830 University Research Court College Park, Maryland 20740

Phone: 301-683-0502 E-mail: @.***

zhanglikate commented 2 years ago

@lipan-NOAA From your job submit directly, I think you are using the new workflow, not the old one.

lipan-NOAA commented 2 years ago

@zhanglikate @rmontuoro @bbakernoaa I have another simulation using workflow (feature/aerosol) and ufs_coupled (feature/p7.1).

Homegfs=/scratch2/NAGAPE/arl/Li.Pan/global-workflow expdir=/scratch2/NCEPDEV/stmp1/Li.Pan/chem202108/firex_1 logdir=/scratch1/NCEPDEV/stmp4/Li.Pan/COM/firex_1/logs/2019070100/gfs.forecast.highres.log

You will see a same error message as last one.

Li

rmontuoro commented 2 years ago

@lipan-NOAA - Since you're running a atmosphere-aerosols case, I'd suggest building the UFS model with the -a flag:

sh build_all.sh -a

Additional driver settings are required in nems.configure when using the fully-coupled version of the model.

lipan-NOAA commented 2 years ago

@rmontuoro I tried it, it had a same failure.

lipan-NOAA commented 2 years ago

figure out the reason, you have to compile the code "sh build_all.sh -a"

noaa-oar-arl / UFS-Aerosol-Config