search

COSIMA / mom6-panan

Pan-Antarctic regional configuration of MOM6
MIT License
6 stars 6 forks source link

Move/copy all inputs to ik11 #37

Closed micaeljtoliveira closed 1 year ago

micaeljtoliveira commented 1 year ago

We should try to have all the inputs in a single location, so they are easy to find and update. This should be in /g/data/ik11/inputs/, just like for the other models/configuratons. Once in a while this directory is copied to tape, so we would also have a backup, which is always a good thing.

I've already created the following directories on ik11:

/g/data/ik11/inputs/mom6/panan/01deg
/g/data/ik11/inputs/mom6/panan/0025deg
/g/data/ik11/inputs/mom6/panan/005deg

Currently 0025deg is empty, but the other two have each a directory named after the date it was created. For the moment, as things are changing quite fast, we can keep updating and adding the files in the existing directories, but later on the idea would be to follow a similar naming and updating scheme as is used for ACCESS-OM2. For ACCESS-OM2, every time a file gets updated, a new directory is created, with links to the unmodified files from the previous directory and with the new files added, replacing the old ones.

Next is a list of all the needed files. Let me know if I missed something. I'm also not sure about the temp_salt_init_z_0025.nc and hycom1_*.nc files. Which ones are needed for each configuration?

01deg - /g/data/ik11/inputs/mom6/panan/01deg/20230404:

005deg - /g/data/ik11/inputs/mom6/panan/005deg/20230404:

0025deg - /g/data/ik11/inputs/mom6/panan/0025deg/20230320:

micaeljtoliveira commented 1 year ago

It would also be good to know if some of the listed files are not needed anymore.

micaeljtoliveira commented 1 year ago

Similarly, we should store the scripts used to generate the topographies in a central location, but probably not ik11.

aekiss commented 1 year ago

Thanks @micaeljtoliveira - re. scripts, we normally keep the canonical versions on github, and note in the metadata of each file what commit was used to generate it.

angus-g commented 1 year ago

I don't think the sponges are needed any more, and I guess it would probably be good to clarify the forcing: we should only need a single boundary forcing file, a single initial condition, and possibly an initial eta? So:

micaeljtoliveira commented 1 year ago

@angus-g Thanks for the explanation.

So the question now is what files should I keep for the 0.1deg configuration:

EDIT: looks like only forcing/forcing_obc.nc and forcing/temp_salt_init_z.nc are mentioned in the MOM_input file, for both the panan-01-hycom1-run and panan-01-zstar-run branches.

EDIT2: hycom1_75_800m.nc is never mentioned in MOM_input, while hycom1_panant.nc is only mentioned in the panan-01-hycom1-run branch.

schmidt-christina commented 1 year ago

I am about to gather all files needed for the 005deg in the folder and was wondering how to go about it.

The two options I am thinking of

  1. Copy files and then change the names of the file to more generic names (i.e. renaming forcing_access_yr2_005.nc from Adele's directory into forcing_obc.nc), but then it's unclear how the files got here, how they were generated and by whom
  2. Link files and use generic names, but then the file is not backed up and could get lost if the owner moves them
micaeljtoliveira commented 1 year ago

I think option 1 is best. To keep track of how files got there, you can for example add a README file. If some of the files required a more complex workflow (e.g., the topography and related files), we should probably follow @aekiss suggestion and put the the script that was used to generated them on github.

micaeljtoliveira commented 1 year ago

I've now gathered all the necessary files for the 0.1deg configuration in a new directory: /g/data/ik11/inputs/mom6/panan/01deg/20230404/

I've tested that it works correctly with both the panan-01-hycom1-run and panan-01-zstar-run branches. Unless someone spots some problem with it, tomorrow I'll update the payu config to use the new directory.

micaeljtoliveira commented 1 year ago

@willaguiar You should now have write permissions on /g/data/ik11/inputs/mom6/panan, so feel free to copy the missing files for the 1/40th configuration whenever you have the opportunity.

willaguiar commented 1 year ago

@willaguiar You should now have write permissions on /g/data/ik11/inputs/mom6/panan, so feel free to copy the missing files for the 1/40th configuration whenever you have the opportunity.

Thanks! I have now copied the initial conditions, and runoff to the 0025 dir. The boundary forcing still has some problem to be fixed, so I will wait to have the final one before moving to the dir.

schmidt-christina commented 1 year ago

I've gathered most of the necessary files for the 0.05deg configuration in the directory: /g/data/ik11/inputs/mom6/panan/005deg/20230404/

But I am still testing and it's not running correctly yet. Will let you know when it works.

schmidt-christina commented 1 year ago

I have copied all files and the 1/20th model has been running for the first 3 days.

I still have one problem:

The filename for the supergrid is hardcoded somewhere and I can't find where. It was named ocean_hgrid_0025.nc, but I now renamed it to ocean_hgrid.nc to be consistent across resolutions. I updated MOM_input and SIS_input.

I get the following error:

`Starting to initialize atmospheric model at 20230405 135938.546

FATAL from PE 536: fms_io_mod(dimesion_size): failed at inquiring size of dimesion nx from file INPUT/ocean_hgrid_0025.nc `

Can someone point me to the file that gets called here? @angus-g @aekiss

For now I am cheating and used ln -s ocean_hgrid.nc ocean_hgrid_0025.nc so that it is running

angus-g commented 1 year ago

It's defined in ocean_mosaic.nc, so you could either change the gridfiles variable in that, or just regenerate the mosaic.

micaeljtoliveira commented 1 year ago

@schmidt-christina The grid filename is hard-coded in ocean_mosaic.nc.

schmidt-christina commented 1 year ago

Thanks, I am using the scripts you provided here (/g/data/x77/ahg157/software), Angus.

I have now changed the gridfiles variable in ocean_mosiac.nc using ncap2 and it's running without problems now.

willaguiar commented 1 year ago

@micaeljtoliveira I am doing some tests runs in the new dir. But the model seem to be unable to read /g/data/ik11/inputs/mom6/panan/0025deg/20230320/masks/*. Could you open reading access to the files in this dir?

micaeljtoliveira commented 1 year ago

@willaguiar The files seem to be readable by everyone:

$ ls -al /g/data/ik11/inputs/mom6/panan/0025deg/20230320/masks
total 76
drwxrwsr-x+ 2 mo1833 ik11  4096 Mar 30 20:56 .
drwxrwsr-x+ 7 mo1833 ik11  4096 Apr  6 12:09 ..
-rw-rw-r--+ 1 mo1833 ik11  6548 Mar 20 10:27 mask_table.1198.141x35
-rw-rw-r--+ 1 mo1833 ik11 14161 Mar 30 20:56 mask_table.2431.204x48
-rw-rw-r--+ 1 mo1833 ik11 14826 Mar 30 15:50 mask_table.2537.202x50
-rw-rw-r--+ 1 mo1833 ik11 18339 Mar 30 20:21 mask_table.3093.220x56
-rw-rw-r--+ 1 mo1833 ik11  2848 Mar 20 10:27 mask_table.580.100x25
-rw-rw-r--+ 1 mo1833 ik11  2946 Mar 20 10:27 mask_table.596.102x25

Same for ACL's, so I'm not sure what else I could do...

angus-g commented 1 year ago

Might be useful to see the actual error message too, @willaguiar?

willaguiar commented 1 year ago

Hi Micael! When I ls /g/data/ik11/inputs/mom6/panan/0025deg/20230320/masks I get that error back:

ls: cannot access '/g/data/ik11/inputs/mom6/panan/0025deg/20230320/masks/mask_table.3093.220x56': Permission denied
ls: cannot access '/g/data/ik11/inputs/mom6/panan/0025deg/20230320/masks/mask_table.2431.204x48': Permission denied
ls: cannot access '/g/data/ik11/inputs/mom6/panan/0025deg/20230320/masks/mask_table.2537.202x50': Permission denied
ls: cannot access '/g/data/ik11/inputs/mom6/panan/0025deg/20230320/masks/mask_table.1198.141x35': Permission denied
ls: cannot access '/g/data/ik11/inputs/mom6/panan/0025deg/20230320/masks/mask_table.596.102x25': Permission denied
ls: cannot access '/g/data/ik11/inputs/mom6/panan/0025deg/20230320/masks/mask_table.580.100x25': Permission denied
mask_table.1198.141x35  mask_table.2431.204x48  mask_table.2537.202x50  mask_table.3093.220x56  mask_table.580.100x25  mask_table.596.102x25

When I run the model setting .../masks/mask_table.3093.220x56as the dir for the masks I get the following error: FATAL from PE 0: MPP_DEFINE_DOMAINS(mpp_compute_extent): domain extents must be positive definite. But if set ../masks_open/ mask_table.3093.220x56 then no error occurs with the masks. (I copied the masks I had on /home/156/wf4500/mom6-panan/input_1_40th/masks2/ to /g/data/ik11/inputs/mom6/panan/0025deg/20230320/masks_open/)

micaeljtoliveira commented 1 year ago

@willaguiar Could you try it now?

willaguiar commented 1 year ago

@willaguiar Could you try it now?

ls now get the correct listing without the permission denied error now. Waiting for the run to start to see if the error goes away

micaeljtoliveira commented 1 year ago

Okay, so the issue was the ACL's on the masks directory, not the files. You only had read and write permissions for the directory. I now added the execute permission.

willaguiar commented 1 year ago

Thanks! Model error FATAL from PE 0: MPP_DEFINE_DOMAINS(mpp_compute_extent): domain extents must be positive definite. disappeared now with the new permission.

willaguiar commented 1 year ago

All files for panan0025 are now in /g/data/ik11/inputs/mom6/panan/0025deg/20230320 with their respective names. Panan0025 is running without any errors with these files. There are two folders with the old forcing (OM2 year 156, not working) and old topography before the edits in the Ross Sea. I'll delete them later today if there are no objections.

micaeljtoliveira commented 1 year ago

@willaguiar Awesome! Thanks

I think we can now close this issue :)