geoschem / GCHP

The "superproject" wrapper repository for GCHP, the high-performance instance of the GEOS-Chem chemical-transport model.
https://gchp.readthedocs.io
Other
22 stars 25 forks source link

[FEATURE REQUEST] Standard GCHP run files #255

Closed sdeastham closed 1 year ago

sdeastham commented 1 year ago

I'd like to propose that users be able to add operational gchp.run examples to the GCHP run directory in runScriptSamples/operational_examples. While I know this is technically already possible, it would be helpful to set a general precedent that this is supported. It should absolutely be the requirement of individual user groups (and not the GCST) to submit pull requests which keep those examples up to date as versions change, but if this is acceptable to the GCST then it would help in keeping users in large groups aligned (especially when the content of gchp.run changes between versions, as it has recently). At MIT, it would for example be a big help to have folders for mit_svante, mit_hex, and mit_engaging (and I am happy to provide the relevant files).

kilicomu commented 1 year ago

I was also going to request this! It would be great for us to be able to keep our environment setup script and a couple of Slurm scripts in there, rather than maintaining separate documentation as I do currently.

lizziel commented 1 year ago

We recently retired the operational examples folder in 14.0 since all non-Harvard scripts were outdated. The original vision was that user groups would submit their scripts to make it easier for users on the same cluster to get started. However, it didn't quite work out since the scripts we received were never updated.

How about we start this up again but this time include a README in each user group folder with contact information, and information about when it was last updated and the cluster it is meant to run on. If someone tries to use the files on the group's system then there is a resource available beyond the GEOS-Chem Support Team. As you say, it would be on the user group to submit updated files. I propose namerunScriptSamples/HPCclusters, with subfolders as suggested by Seb (mit_svante, harvard_cannon, nasa_pleiades, etc).

Thoughts?

kilicomu commented 1 year ago

Happy with that. I can PR something with York info in the near future if that helps to get the ball rolling. Do we need the level of HPCclusters or can we just put site-specific examples directly in their own directories in runScriptSamples, e.g.

runScriptSamples/york_viking
sdeastham commented 1 year ago

Agreed - I like this plan. I'm open to either with or without HPCclusters - whatever the GCST thinks will work best. I think with all the great changes that have gone into GCHP's run directory, the scripts should be much easier to keep up to date (it certainly looks like gchp.run is much shorter now).

lizziel commented 1 year ago

Great. @yantosca, @msulprizio, @Jourdan-He - do you have a preference about if the specific cluster folders should be in a dedicated subdirectory? I suggested it just to keep the runScriptSamples root level tidy. I don't anticipate getting file submissions for many clusters, but you never know over time.

msulprizio commented 1 year ago

We still seem to have the operational_examples directory in 14.0.0-rc.3 so maybe it wasn't retired? That directory contains subdirectories for adjoint, harvard_cannon, nasa_pleiades, and wustl_gcst. See https://github.com/geoschem/geos-chem/tree/dev/run/GCHP/runScriptSamples/operational_examples. If we keep that as-is, I don't think there's a need for an HPCclusters directory. Other groups can simply add their own directories there and submit them via PRs.

Jourdan-He commented 1 year ago

I agree with @msulprizio , the current location of wustl_gcst folder makes sense.

msulprizio commented 1 year ago

I agree with @msulprizio , the current location of wustl_gcst folder makes sense.

My only suggestion there is that we rename that particular directory to wustl_compute1 following the naming scheme suggested above (i.e. institution_clustername).

lizziel commented 1 year ago

Oh geez, sorry for the spread of misinformation! So maybe all that is needed is better advertising for users to submit their files.

lizziel commented 1 year ago

This issue is addressed in https://github.com/geoschem/geos-chem/pull/1443 which is going into 14.0.1. @sdeastham, @kilicomu, please send your files whenever is best for you and we will include them in whatever version is coming next at the time.