Confusing BASELINE result: PASS but also "Could not interpret CPRNC output"

[billsacks]

I'm confused by the TestStatus I'm seeing in a test I just ran with baseline comparisons:

PASS SMS_Ld2.ne30pg3_t232.BLT1850.derecho_intel.allactive-defaultio BASELINE cesm3_0_alpha03c.derecho: Could not interpret CPRNC output

There are two things that I find confusing about this:

(1) It's not clear to me what CPRNC output is leading to this "Could not interpret" message. I'm attaching the TestStatus.log file here. At a glance, it looks like all cprnc output is showing typical "IDENTICAL" lines.

TestStatus.log

(2) If it's really the case that there's at least one CPRNC file for which the code can't interpret the output, I feel like it would be safest to mark the result as a FAIL rather than a PASS, so that this isn't overlooked.

It looks like @jasonb5 introduced this in https://github.com/ESMCI/cime/commit/e33bedee161b8ba844b7109420799cb0316cba52, so Jason, I'm assigning this to you in hopes that you can help clarify this for me.

[billsacks]

I reran this and got the same message. I'm also seeing the same message in a lot of the tests that @fischer-ncar has been running recently.

[fischer-ncar]

I'm seeing these messages too. But didn't notice them since it was the BASELINE was passing.

[jedwards4b]

Comments passed to hist_utils:get_ts_synopsis are

Comparing hists for case 'SMS_Ld5.f09_f09_mg17.PC6.derecho_intel.cam-cam6_port_f09.C.20240927_080217_54xyj4' dir1='/glade/derecho/scratch/jedwards/SMS_Ld5.f09_f09_mg17.PC6.derecho_intel.cam-cam6_po\
rt_f09.C.20240927_080217_54xyj4/run', suffix1='',  dir2='/glade/campaign/cesm/cesmdata/cesm_baselines/cesm3_0_alpha03d/SMS_Ld5.f09_f09_mg17.PC6.derecho_intel.cam-cam6_port_f09' suffix2=''           
  comparing model 'cam'                                                                                                                                                                               
    SMS_Ld5.f09_f09_mg17.PC6.derecho_intel.cam-cam6_port_f09.C.20240927_080217_54xyj4.cam.h0a.0001-01-01-00000.nc matched cam.h0a.0001-01-01-00000.nc                                                 
    SMS_Ld5.f09_f09_mg17.PC6.derecho_intel.cam-cam6_port_f09.C.20240927_080217_54xyj4.cam.h0i.0001-01-01-00000.nc matched cam.h0i.0001-01-01-00000.nc                                                 
    SMS_Ld5.f09_f09_mg17.PC6.derecho_intel.cam-cam6_port_f09.C.20240927_080217_54xyj4.cam.h1i.0001-01-01-00000.nc matched cam.h1i.0001-01-01-00000.nc                                                 
PASS        

I see no reason for this to trigger the "Could not interpret CPRNC output" - except that as far as I can tell the comment should not have been passed in at all? @jasonb5

[jasonb5]

I see the issue the catch-all is Could not interpret CPRNC output so even if diff_test: the two files seem to be IDENTICAL is present it will always return a synopsis that is not empty. Not the intended behavior, I'll have a fix shortly.

[billsacks]

Thanks a lot @jasonb5 !