search

ACCESS-NRI / ACCESS-OM2

ACCESS-OM2: ACCESS Ocean-Sea Ice Model
Apache License 2.0
5 stars 0 forks source link

Documentation on custom builds #58

Open aekiss opened 3 months ago

aekiss commented 3 months ago

Is there clear, step-by-step documentation on how users can

There are some users (e.g. @kialstewart) who will need to build their own executables, but I couldn't find this info documented anywhere. Apologies if I didn't look carefully enough...

aidanheerdegen commented 3 months ago

Sorry Andrew, only just seen this. Thanks for bringing this up.

Will respond with something of substance ASAP.

aekiss commented 1 month ago

Thanks @aidanheerdegen et al for all your work on documenting this.

I couldn't find anything on my first point - apologies if I overlooked it

  • find the MOM5, CICE5, libaccessom2 source code used by a given executable

-- is there a way to go from an exe entry in a config.yaml to the corresponding commit in the source repository for that model component? This is something users need quite frequently, to see exactly what code they're running. If it's complicated to do that with Spack, perhaps the configs could contain a tool (e.g. python script) that automates it by reading config.yaml and doing the required spack magic? Or perhaps README.md for each config could include links to the specific commits of MOM5, CICE5 and libaccessom2, auto-updated by CI?

aidanheerdegen commented 1 month ago

You're right, I didn't address your question about finding the source code used by an executable.

We're now capturing all the build information from deployed models

https://github.com/ACCESS-NRI/build-cd/issues/33

The intention is to expose this information in an experiment metadata view, but also make it viewable standalone as record of all builds. A initial prototype that @utkarshgupta95 just created on Friday is available here:

https://experiment-metadb-seven.vercel.app/release-provenance

You can see the individual components in each model by toggling the arrow in the left most "components" column.

aekiss commented 1 month ago

Great! This is awesome.

Some suggestions:

  1. is it possible to go to a particular view of https://experiment-metadb-seven.vercel.app/release-provenance (ie the view you get when clicking the toggle arrow) via a direct URL? That would make it easier to link to the relevant table entry.
  2. following from 1, this direct link to the table entry could be put in the README for each config
  3. it would be one less click to see the source if the table links to the tree view rather than a release tag, eg https://github.com/ACCESS-NRI/MOM5/tree/2023.11.09 rather than https://github.com/ACCESS-NRI/mom5/releases/tag/2023.11.09
aidanheerdegen commented 1 month ago

is it possible to go to a particular view of https://experiment-metadb-seven.vercel.app/release-provenance (ie the view you get when clicking the toggle arrow) via a direct URL? That would make it easier to link to the relevant table entry.

Should be possible. I'll ask. The intention was that we can excerpt a subset of model versions and just display those so that it is flexible, which covers your use case I think.

following from 1, this direct link to the table entry could be put in the README for each config

Yes I suppose it could. Good idea.

it would be one less click to see the source if the table links to the tree view rather than a release tag

Good point. I'll ask @utkarshgupta95 to make those changes.

aidanheerdegen commented 1 month ago

@aekiss The first iteration of the developer docs have been merged and are available here

https://github.com/ACCESS-NRI/ACCESS-OM2/blob/main/DEVELOPERS.md

This covers only the most basic use-case of altering a single model component, and doesn't cover the following use cases:

access-hive-bot commented 1 month ago

This issue has been mentioned on ACCESS Hive Community Forum. There might be relevant details there:

https://forum.access-hive.org.au/t/developer-docs/2133/1

aekiss commented 1 month ago

Great, thanks for the documentation @aidanheerdegen and team, hopefully I'll have a chance to look through it again next week.

A couple more thoughts:

aidanheerdegen commented 1 month ago

will it be possible for users to use the release provenance tool to keep track of the sources used in their local custom builds? Or will it only support the ACCESS-NRI official releases?

Currently this is only capturing builds deployed by ACCESS-NRI.

can the component repo git hash be included in each component's executable name or path to help users track provenance of their custom builds, as we're doing with ACCESS-OM3?

Micael's solution required specifying the exact commit to install. I don't know how that will work for a developer workflow which isn't doing this. The developer workflow specifies a version to clone and after that it will built whatever is in the local source clone, regardless if it has been git committed or not. Isn't that the same situation that existed with the COSIMA ACCESS-OM2 build scripts?

We could provide some guidance for developers about best practices for building their modified code and providing build provenance. e.g. committing their changes and pushing to a fork and building directly from there, and/or adding their spack.lock file to their model repo.

aidanheerdegen commented 1 month ago

Something that is missing from the first version of the developer docs is how to clean up when development has paused, or finished, i.e. spack env deactivate