Template computer/code setup and new code registry configs

unkcpz commented 2 years ago

(Added to https://github.com/aiidateam/team-compass/issues/4)

Backgroud

For beginners and even for the experienced AiiDA user, setting up computers and codes is still a tedious mission. If using the interactive mode, although it is good that options are prompted up and the user can set every option one by one carefully, it requires going through all options even if some are not necessary and time-consuming for a similar setup that has shared options with other code/computer setup.

AiiDA provides the non-interactive mode to set up the computer/code from a config YAML file, which lowers the burden for users who need to set up the computer/code next time. However, the non-interactive mode requires a YAML file as the input, and not clear which options are mandatory, and let alone it is not clear which default value will be used without checking the command help message or even the source code. Let alone for the computer setup it is a two-stage process, user needs to set up the computer for attributes that are common information for the computer that is stored in the database using verdi computer setup. Then running verdi computer configuration to set up information of the computer that is specific to the user or required to modify after the node is stored in the database.

The computer/code can be set up from a YAML file, and we provide a repository aiida-code-registry to store the YAML files for public computers and codes to share with others. Need to mention that the interactive setup command can accept a URL of a remote YAML file for setup. This makes it possible to not download/clone the aiida-code-registry repo to use the YAML to set up computer/code.

Motivation and impact

As a user, I want to have a way to easily setup computer/code and start using AiiDA quickly.

As a user, I want to go to a centralized place to share or find the configurations of my settings.

As a user, when I set the for the same computer and codes for another AiiDA profile, I can do it without duplicating all processes but very easily accomplish it.

As an aiida-code-registry developer, I need all the config sharing processes automatically and put less effort on review the new computer/code configs added in the repo.

As an aiidalab-widgets-base developer, I want to use a official supported template system to set up computer/code for ComputationalResourceWidget for setting computer/code from "database".

Desired outcome

Setup computer/code from template foramt config files
Registry page to show/share new configure files

Complexity

The template system need to be supported with verdi, which the DynamicEntryPointCommandGroup can be potentially used to prompt the required parameters dynamically for user's input. But it is not clear how to make it works.

The computer/code registry page need redesigned and well documented in the AiiDA-core docs to encourage users to contribute and maintain it.

Further comments (by GP)

@csadorf @unkcpz you are using this for the AiiDAlab right? Can you please comment here on what is the status and if/what are the missing actions? E.g.:

is the repo easily findable for a generic AiiDA user, e.g. from the AiIDA-core docs?
are there clear instructions on how to contribute a new code/computer?
are there outstanding technical issues, e.g. how to always ask to customize some aspects of a computer such as account or allocation or project?
are at least the (public, i.e. not private inside a university) computers/codes we use there? E.g. Daint, Eiger, Lumi-C, ...
can we identify a person in charge of maintaining the repo?

I think some of these are in the issues of the repo https://github.com/aiidateam/aiida-code-registry, but probably not all of them. If we find a person in charge, this person can triage a bit the tasks and prioritise them (and possibly distribute them, e.g. organising a "coding day"?)

csadorf commented 2 years ago

@giovannipizzi Yes, this is used in the context of AiiDAlab. I do not know who the primary maintainer is, in case that there is one.

unkcpz commented 2 years ago

If we find a person in charge, this person can triage a bit the tasks and prioritise them (and possibly distribute them, e.g. organising a "coding day"?)

I think what @ltalirz did on https://github.com/aiidateam/aiida-code-registry/pull/62 is related but not exactly what we want to have in the end. Maybe @ltalirz and I or anyone interesting in having this can have a short meeting to separate tasks on what we want to achieve at the end for 1. easy to configure computer/code with verdi. 2. Templates as computer/code database for aiidalab to easily configure through ComptuerResource widget.

sphuber commented 1 year ago

See https://github.com/aiidateam/team-compass/issues/4

aiidateam / aiida-code-registry