search

NOAA-EMC / GSI

Gridpoint Statistical Interpolation
GNU Lesser General Public License v3.0
64 stars 146 forks source link

latest gsi develop version has trouble reading hirs4_metop-a.SpcCoeff.bin from crtm/2.4.0.1 #752

Open jack-woollen opened 1 month ago

jack-woollen commented 1 month ago

@jswhit @ADCollard Error message below from reanalysis merge with latest develop. Is there a new crtm version for this?

GLBSOI: jiter,jiterstart,jiterlast,jiterend= 1 1 2 1 SpcCoeff_ReadFile(Binary)(FAILURE) : Error reading channel data. input statement requires too much data, unit 10, file /lfs/h2/emc/ptmp/Jack.Woollen/runda/cfsv2/cdas1.163176/./crtm_coeffs/hirs4_metop-a.SpcCoeff.bin CRTM_SpcCoeff_Load(FAILURE) : Error reading SpcCoeff file #1, ./crtm_coeffs/hirs4_metop-a.SpcCoeff.bin; Process ID: 88 CRTM_Init(FAILURE) : Error loading SpcCoeff data; Process ID: 88 crtm_interface*init_crtm: ERROR crtm_init error_status= 3 TERMINATE PROGRAM EXECUTION MPICH ERROR [Rank 88] [job id 7ebc1672-c444-42a5-af33-272592e0b982] [Fri Jun 7 10:23:17 2024] [nid001178] - Abort(71) (rank 88 in comm 0): application called MPI_Abort(MPI_COMM_WORLD, 71) - process 88

dtkleist commented 1 month ago

I don't thinks so. @emilyhcliu -- any insight?

emilyhcliu commented 1 month ago

Daryl,

The coefficients for HIRS4 is messed up in CRTM 2.4.0.1. We do not use HIRS4/HIRS3 anymore. We should remove all HIRS related observation input from the GSI namelist in the analysis script.

Emily

Emily Huichun Liu Physical Scientist Modeling and Data Assimilation Branch - DA & QC NOAA/NWS/NCEP Environmental Modeling Center National Centers for Weather and Climate Prediction (NCWCP) 5830 University Research Court College Park, MD 20740 Phone: 301-683-3639 Cell: 410-948-6256

On Fri, Jun 7, 2024 at 11:35 AM Daryl Kleist @.***> wrote:

I don't thinks so. @emilyhcliu https://github.com/emilyhcliu -- any insight?

— Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/GSI/issues/752#issuecomment-2155081496, or unsubscribe https://github.com/notifications/unsubscribe-auth/AITLO5W6SDRNBZXAIPYMY5DZGHHNXAVCNFSM6AAAAABI6NX4P6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNJVGA4DCNBZGY . You are receiving this because you were mentioned.Message ID: @.***>

dtkleist commented 1 month ago

Emily -- that's true for real time / default namelist, but we need to have a fix for reanalysis efforts. Daryl

On Fri, Jun 7, 2024 at 1:05 PM emilyhcliu @.***> wrote:

Daryl,

The coefficient name for HIRS4 is messed up in CRTM 2.4.0.1. We do not use HIRS4/HIRS3 anymore. We should remove all HIRS related observation input from the GSI namelist in the analysis script.

Emily

Emily Huichun Liu Physical Scientist Modeling and Data Assimilation Branch - DA & QC NOAA/NWS/NCEP Environmental Modeling Center National Centers for Weather and Climate Prediction (NCWCP) 5830 University Research Court College Park, MD 20740 Phone: 301-683-3639 Cell: 410-948-6256

On Fri, Jun 7, 2024 at 11:35 AM Daryl Kleist @.***> wrote:

I don't thinks so. @emilyhcliu https://github.com/emilyhcliu -- any insight?

— Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/GSI/issues/752#issuecomment-2155081496, or unsubscribe < https://github.com/notifications/unsubscribe-auth/AITLO5W6SDRNBZXAIPYMY5DZGHHNXAVCNFSM6AAAAABI6NX4P6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNJVGA4DCNBZGY>

. You are receiving this because you were mentioned.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/GSI/issues/752#issuecomment-2155214101, or unsubscribe https://github.com/notifications/unsubscribe-auth/AETTYFBKOX5FTVEU6EXPB53ZGHR6FAVCNFSM6AAAAABI6NX4P6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNJVGIYTIMJQGE . You are receiving this because you commented.Message ID: @.***>

emilyhcliu commented 1 month ago

Daryl,

I checked the global-workfow develop. We do not include HIRS sensors in the GSI analysis script (script/exglobal_atmos_analysis.sh) anymore.

Emily Emily Huichun Liu Physical Scientist Modeling and Data Assimilation Branch - DA & QC NOAA/NWS/NCEP Environmental Modeling Center National Centers for Weather and Climate Prediction (NCWCP) 5830 University Research Court College Park, MD 20740 Phone: 301-683-3639 Cell: 410-948-6256

On Fri, Jun 7, 2024 at 1:05 PM Emily Liu - NOAA Federal @.***> wrote:

Daryl,

The coefficient name for HIRS4 is messed up in CRTM 2.4.0.1. We do not use HIRS4/HIRS3 anymore. We should remove all HIRS related observation input from the GSI namelist in the analysis script.

Emily

Emily Huichun Liu Physical Scientist Modeling and Data Assimilation Branch - DA & QC NOAA/NWS/NCEP Environmental Modeling Center National Centers for Weather and Climate Prediction (NCWCP) 5830 University Research Court College Park, MD 20740 Phone: 301-683-3639 Cell: 410-948-6256

On Fri, Jun 7, 2024 at 11:35 AM Daryl Kleist @.***> wrote:

I don't thinks so. @emilyhcliu https://github.com/emilyhcliu -- any insight?

— Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/GSI/issues/752#issuecomment-2155081496, or unsubscribe https://github.com/notifications/unsubscribe-auth/AITLO5W6SDRNBZXAIPYMY5DZGHHNXAVCNFSM6AAAAABI6NX4P6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNJVGA4DCNBZGY . You are receiving this because you were mentioned.Message ID: @.***>

emilyhcliu commented 1 month ago

CRTM team knows about the issue in HIRS coefficients.
I will contact CRTM team to get their plans about fixing HIRS coefficients or check if there is a newer version with HIRS coefficiets fixed.

Will repot back to here.

ADCollard commented 1 month ago

@emilyhcliu @jack-woollen @dtkleist Sorry I have not been checking e-mails the last couple of days. As @emilyhcliu said, this is a versioning issue with the HIRS coefficient files that came to light when the code needed to be updated to use more recent ATMS files.

I actually mentioned it in this CRTM PR , but it needs an explicit issue (depending on whether Emily had any luck with the CRTM team).

It might be possible to edit the HIRS files ourselves using the (possibly obsolete) tools in the CRTM. I can take a look at that on Monday.

One question, is there a requirement for reanalysis to use the operational installation of the CRTM fix files or could be point to another place for HIRS? I ask as, even if we are able to update the fix files, NCO has been insisting on a complete re-install of CRTM just to update fix files and that takes months.

ADCollard commented 3 weeks ago

I used a modified version of SpcCoeff_Inspect.f90 to modify the version numbers that were causing the problems.

The modified files are on Hera at /scratch1/NCEPDEV/da/Andrew.Collard/git/CRTM_REL-2.4.0_emc/fix/SpcCoeff/Big_Endian/hirs_fix (big endian only)