search

ESMCI / cime

Common Infrastructure for Modeling the Earth
http://esmci.github.io/cime
Other
161 stars 206 forks source link

Enhancement request: Use realpath when comparing namelists #4638

Open samsrabin opened 4 months ago

samsrabin commented 4 months ago

When comparing a test to a baseline generated by a different user, I ran into the following supposed diff in our namelists:

SMS_D_Ld5.5x5_amazon.I1850Clm60Bgc.derecho_gnu.clm-HillslopeC
2024-05-28 11:36:12: NLCOMP
Comparison failed between '/glade/derecho/scratch/samrabin/tests_0528-113433de/SMS_D_Ld5.5x5_amazon.I1850Clm60Bgc.derecho_gnu.clm-HillslopeC.GC.0528-113433de_gnu/CaseDocs/nuopc.runconfig' with '/glade/campaign/cgd/tss/ctsm_baselines/ctsm5.2.005/SMS_D_Ld5.5x5_amazon.I1850Clm60Bgc.derecho_gnu.clm-HillslopeC/CaseDocs/nuopc.runconfig'
DRIVER_attributes->PELAYOUT_attributes->ALLCOMP_attributes:
 - mesh_atm : /glade/campaign/cesm/cesmdata/cseg/inputdata/lnd/clm2/testdata/ESMFmesh_10x15_synthetic_cosphill_1.0.nc
 + mesh_atm : /glade/campaign/cesm/cesmdata/inputdata/lnd/clm2/testdata/ESMFmesh_10x15_synthetic_cosphill_1.0.nc
DRIVER_attributes->PELAYOUT_attributes->ALLCOMP_attributes:
 - mesh_lnd : /glade/campaign/cesm/cesmdata/cseg/inputdata/lnd/clm2/testdata/ESMFmesh_10x15_synthetic_cosphill_1.0.nc
 + mesh_lnd : /glade/campaign/cesm/cesmdata/inputdata/lnd/clm2/testdata/ESMFmesh_10x15_synthetic_cosphill_1.0.nc

I say "supposed" because the differences are only skin-deep: The new versions (+) resolve through an alias to match the baseline versions (-) exactly.

There may be reasons it is the way it is, but I feel like it'd be better for such situations to not count as diffs. I think this could be accomplished by using os.path.realpath() somewhere in _normalize_string_value(). Although looking at that code, I see that os.path.basename() is used, so maybe I'm misunderstanding what's happening where.

github-actions[bot] commented 1 month ago

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days.

jedwards4b commented 1 month ago

@samsrabin I think that this may have just been a problem on one machine due to sym-links - is there still an issue?

samsrabin commented 1 month ago

There may well still be an issue, yes—I haven't recently been in the situation where the problem arises.

ekluzek commented 1 month ago

This does have to do with softlinks. The underlying reason it came up was that I was inadvertently overriding $CESMDATAROOT in my startup files (from way in the past). Now, I've stopped doing that so we aren't likely to see it anymore. This was on Derecho and not that long ago though.

So it's a good change to make, but might not be worth it. Also you could argue that the components that come up with the namelist filenames is the place where this should be corrected. The problem there is it has to be done in each component. If this was easy to do in cime it would allow it to be done in one place.