0.25deg input files

[aekiss]

This 0.25deg filename

/g/data/ik11/inputs/access-om3/0.x.0/025deg/share/access-om2-nomask-025deg-ESMFmesh.nc

is inconsistent with 1deg

/g/data/ik11/inputs/access-om3/0.x.0/1deg/share/access-om2-1deg-nomask-ESMFmesh.nc

so should be renamed as

/g/data/ik11/inputs/access-om3/0.x.0/025deg/share/access-om2-025deg-nomask-ESMFmesh.nc

[aekiss]

The unnecessary input files should also be removed from /g/data/ik11/inputs/access-om3/0.x.0/025deg at some point - it has more than double the number of files compared to 1deg:

$ tree /g/data/ik11/inputs/access-om3/0.x.0/025deg
/g/data/ik11/inputs/access-om3/0.x.0/025deg
├── cice
│   ├── global_gx1v5_grid.nc
│   ├── grid.nc
│   ├── i2o.nc
│   ├── kmt.nc
│   ├── monthly_sstsss.nc
│   ├── o2i.nc
│   └── u_star.nc
├── mom
│   ├── chl.nc
│   ├── mask_table.464.48x40
│   ├── ocean_hgrid.nc
│   ├── ocean_mask.nc
│   ├── ocean_mask_table
│   ├── ocean_mosaic.nc
│   ├── ocean_temp_salt.res.nc
│   ├── ocean_vgrid.nc
│   ├── roughness_amp.nc
│   ├── roughness_cdbot.nc
│   ├── salt_sfc_restore.nc
│   ├── tideamp.nc
│   └── topog.nc
└── share
    ├── access-om2-025deg-ESMFmesh.nc
    ├── access-om2-nomask-025deg-ESMFmesh.nc
    ├── grid.nc
    ├── meshes
    │   └── JRA55do-ESMFmesh.nc
    ├── ocean_hgrid.nc
    ├── ocean_mask.nc
    └── topog.nc

4 directories, 27 files

$ tree /g/data/ik11/inputs/access-om3/0.x.0/1deg
/g/data/ik11/inputs/access-om3/0.x.0/1deg
├── cice
│   ├── grid.nc
│   ├── iced.1900-01-01-10800.nc
│   └── kmt.nc
├── mom
│   ├── ocean_hgrid.nc
│   ├── ocean_temp_salt.res.nc
│   ├── ocean_vgrid.nc
│   └── salt_sfc_restore.nc
├── share
│   ├── access-om2-1deg-ESMFmesh.nc
│   ├── access-om2-1deg-nomask-ESMFmesh.nc
│   └── topog.nc
└── ww3
    ├── mod_def.ww3
    └── restart.ww3

4 directories, 12 files

[anton-seaice]

It would be nice if the two ESMF mesh files had full paths for the ocean files in the attributes too.

From ncdump:

:inputFile = "ocean_hgrid.nc, ocean_mask.nc" ;
:history = "Created using: python3 /g/data/tm70/ek4684/inputs/access-om3/0.x.0/025deg/share/generate_mesh.py --grid-type=mom --grid-filename=ocean_hgrid.nc --mesh-filename=access-om2-025deg-ESMFmesh.nc --mask-filename=ocean_mask.nc --wrap-lons" ;

[ezhilsabareesh8]

I have incorporated all the changes including full paths for the ocean files in the attributes.

/g/data/ik11/inputs/access-om3/0.x.0/025deg/
├── cice
│   ├── grid.nc
│   └── kmt.nc
├── mom
│   ├── ocean_hgrid.nc
│   ├── ocean_temp_salt.res.nc
│   ├── ocean_vgrid.nc
│   ├── salt_sfc_restore.nc
│   └── topog.nc
└── share
    ├── access-om2-025deg-ESMFmesh.nc
    ├── access-om2-025deg-nomask-ESMFmesh.nc
    └── topog.nc

netcdf access-om2-025deg-ESMFmesh {
dimensions:
    nodeCount = 1555447 ;
    coordDim = 2 ;
    elementCount = 1555200 ;
    maxNodePElement = 4 ;
variables:
    double nodeCoords(nodeCount, coordDim) ;
        nodeCoords:units = "degrees" ;
    int elementConn(elementCount, maxNodePElement) ;
        elementConn:long_name = "Node indices that define the element connectivity" ;
    int numElementConn(elementCount) ;
        numElementConn:long_name = "Number of nodes per element" ;
    double centerCoords(elementCount, coordDim) ;
        centerCoords:units = "degrees" ;
    byte elementMask(elementCount) ;
    double elementArea(elementCount) ;

// global attributes:
        :gridType = "unstructured mesh" ;
        :timeGenerated = "2024-03-26 17:35:06.858546" ;
        :created_by = "ek4684" ;
        :inputFile = "/g/data/ik11/inputs/access-om2/input_20230515_025deg_topog/mom_025deg/ocean_hgrid.nc, /g/data/ik11/inputs/access-om2/input_20230515_025deg_topog/mom_025deg/ocean_mask.nc" ;
        :history = "Created using: python3 /g/data/tm70/ek4684/om3-scripts/mesh_generation/generate_mesh.py --grid-type=mom --grid-filename=/g/data/ik11/inputs/access-om2/input_20230515_025deg_topog/mom_025deg/ocean_hgrid.nc --mesh-filename=access-om2-025deg-ESMFmesh.nc --mask-filename=/g/data/ik11/inputs/access-om2/input_20230515_025deg_topog/mom_025deg/ocean_mask.nc --wrap-lons" ;
}

[aekiss]

Great, thanks @ezhilsabareesh8!

The file paths for access-om2-025deg-nomask-ESMFmesh.nc in the config will also need to be fixed of course https://github.com/COSIMA/MOM6-CICE6/compare/1deg_jra55do_ryf...025deg_jra55do_ryf_iss101#diff-1e49cda0a4f3d63dbbd922d4e8f99c370ac40fe05c4187795d4c4bde35719829

[aekiss]

In the attributes it would be good to include hashes for the input files and the git commit of om3-scripts/mesh_generation/generate_mesh.py to really nail it down precisely

[ezhilsabareesh8]

The file paths for access-om2-025deg-nomask-ESMFmesh.nc in the config will also need to be fixed of course COSIMA/MOM6-CICE6@1deg_jra55do_ryf...025deg_jra55do_ryf_iss101#diff-1e49cda0a4f3d63dbbd922d4e8f99c370ac40fe05c4187795d4c4bde35719829

I have updated the file names in the config in this commit

[dougiesquire]

In the attributes it would be good to include hashes for the input files and the git commit of om3-scripts/mesh_generation/generate_mesh.py to really nail it down precisely

The commit is added if the command is run from the git repo, e.g.:

ncdump -h /g/data/ik11/inputs/access-om3/0.x.0/1deg/share/access-om2-1deg-ESMFmesh.nc
netcdf access-om2-1deg-ESMFmesh {
dimensions:
    nodeCount = 108071 ;
    coordDim = 2 ;
    elementCount = 108000 ;
    maxNodePElement = 4 ;
variables:
    double nodeCoords(nodeCount, coordDim) ;
        nodeCoords:units = "degrees" ;
    int elementConn(elementCount, maxNodePElement) ;
        elementConn:long_name = "Node indices that define the element connectivity" ;
    int numElementConn(elementCount) ;
        numElementConn:long_name = "Number of nodes per element" ;
    double centerCoords(elementCount, coordDim) ;
        centerCoords:units = "degrees" ;
    byte elementMask(elementCount) ;
    double elementArea(elementCount) ;

// global attributes:
        :gridType = "unstructured mesh" ;
        :timeGenerated = "2024-01-25 15:01:40.899669" ;
        :created_by = "ds0092" ;
        :inputFile = "/g/data/ik11/inputs/access-om2/input_20201102/mom_1deg/ocean_hgrid.nc, /g/data/ik11/inputs/access-om2/input_20201102/mom_1deg/ocean_mask.nc" ;
        :history = "Created using commit 7b06e94dc79684c8e5b0d33a36d8046d42e9bc95 of mesh_generation/generate_mesh.py in git@github.com:COSIMA/om3-scripts.git: python3 /g/data/tm70/ds0092/model/om3-scripts/mesh_generation/generate_mesh.py --grid-type=mom --grid-filename=/g/data/ik11/inputs/access-om2/input_20201102/mom_1deg/ocean_hgrid.nc --mesh-filename=/g/data/ik11/inputs/access-om3/0.x.0/1deg/share/access-om2-1deg-ESMFmesh.nc --mask-filename=/g/data/ik11/inputs/access-om2/input_20201102/mom_1deg/ocean_mask.nc --wrap-lons" ;

I'll fix this up so it works more generally and think about adding hashes also.

It would be nice if the two ESMF mesh files had full paths for the ocean files in the attributes too.

Yes, I'll fix this too.

[anton-seaice]

It would be good to also have the md5sum for ocean_hgrid.nc and ocean_mask.nc files.

[anton-seaice]

You could do it using this : https://github.com/COSIMA/om3-utils/blob/6b8c64498a3213e9b31603d989a5fcfb5f843132/om3utils/utils.py

f.inputfile_md5 = md5sum(ocean_hgrid)

[dougiesquire]

FYI, with https://github.com/COSIMA/om3-scripts/pull/10 input file hashes and full paths are added to the mesh file metadata. The parsing of the git url will now also still work if the script is called from outside the repo.

[dougiesquire]

@ezhilsabareesh8 can this be closed now?

[ezhilsabareesh8]

Thanks @dougiesquire, yes this can be closed