ACCESS-NRI / access-hive.org.au

Documentation hub for the Earth System models, ACCESS, and their community
https://access-hive.org.au/
8 stars 12 forks source link

Make turn on sync in run access-om2 instructions more obvious #699

Closed anton-seaice closed 5 days ago

anton-seaice commented 2 weeks ago

@pwongpan has been using the "Run ACCESS-OM2" instructions are suggests that turning on "Sync" should be a more obvious step in the instructions.

It is covered here, which is towards the bottom of the page:

https://access-hive.org.au/models/run-a-model/run-access-om/#syncing-output-data

But its a fairly essential step, so we should make it a new step between "Get ACCESS-OM2 configuration" and "Run ACCESS-OM2 configuration"

pwongpan commented 2 weeks ago

Thanks @anton-seaice. I will continue to provide feedback on other sections that I find may be useful, too.

atteggiani commented 1 week ago

@pwongpan has been using the "Run ACCESS-OM2" instructions are suggests that turning on "Sync" should be a more obvious step in the instructions.

It is covered here, which is towards the bottom of the page:

https://access-hive.org.au/models/run-a-model/run-access-om/#syncing-output-data

But its a fairly essential step, so we should make it a new step between "Get ACCESS-OM2 configuration" and "Run ACCESS-OM2 configuration"

Would that be ok if we put a note (visible with a different colour) in the Run ACCESS-OM2 configuration paragraph before the actual Run configuration subparagraph, that links to the Syncing output data paragraph below? What do you think can be placed in the text of the note?

pwongpan commented 1 week ago

@atteggiani @anton-seaice that's great idea. In addition, what do you think about adding the "offline" instruction to merge the daily files into a single netcdf_daily file after the payu run finishes, e.g., for some test runs?

atteggiani commented 1 week ago

@atteggiani @anton-seaice that's great idea. In addition, what do you think about adding the "offline" instruction to merge the daily files into a single netcdf_daily file after the payu run finishes, e.g., for some test runs?

Thank you for the feedback @pwongpan! Sorry, I am not too familiar with ACCESS-OM2 and the workflow to merge daily files into a single netCDF. Is there a script for that step? Does ACCESS-NRI have a release of that @anton-seaice?

anton-seaice commented 1 week ago

There is a payu userscript to do the merge of daily data automatically, but it runs at the sync step.

So if we turn on sync, then the merge step "should" happen automatically.

pwongpan commented 1 week ago
# Sync options for automatically copying data from ephemeral scratch space to
# longer term storage
sync:
    enable: False # set path below and change to true
    path: null # Set to location on /g/data or a remote server and path (rsync syntax)

Does that mean I set enable: True path: null

Or should I change path to match with shortpath in the script too?

Thanks again both.

anton-seaice commented 1 week ago

Almost ... set path to be a location on /g/data/ where you want the experiment to persist:

enable:True
path: /g/data/<PROJECT>/<USER>/<Experiment_Name>
atteggiani commented 1 week ago

There is a payu userscript to do the merge of daily data automatically, but it runs at the sync step.

So if we turn on sync, then the merge step "should" happen automatically.

I think it is important to flag this in the note or in the sync paragraph, so that users know.

pwongpan commented 1 week ago

Thanks both. I may suggest we add something like "Note, however, that the size of /g/data/ is potentially limited depending on the Project you are associated with".

atteggiani commented 1 week ago

I opened a PR #703 to add what discussed above. On the PR page there is a link for the deployment of the website to check a preview.

@anton-seaice I set you as a reviewer. @pwongpan feel free to comment if you have any better suggestions for the warning text or to make it clearer.