search

idaholab / moose

Multiphysics Object Oriented Simulation Environment
https://www.mooseframework.org
GNU Lesser General Public License v2.1
1.71k stars 1.04k forks source link

Create an Application "Capability Registry" #26470

Open dschwen opened 8 months ago

dschwen commented 8 months ago

Reason

We have a ton of custom logic in the test harness to conditionally skip tests. We should move the information that logic is based on into the executable. Some functionality is contingent on optional libraries. Currently we test if the library submodule is initialized, but that test alone is insufficient in both directions. Optional libraries can be supplied out of tree using the MYLIBRAY_DIR env variable pattern, and optional libraries might themselves depend on additional libraries (or may be architecture dependent etc.). This can result in bot false positives and false negatives.

Design

I propose a capability registry on MooseApp that can be dumped using a command line argument --show_capabilities. This option will dump a JSON blurb that the test harness can parse. I suggest using a dictionary that can have bool, string, or number types.

On the test harness side lines like

libtorch = true
required_submodule = 'contrib/thermochimica'

will turn, into

capabilities = 'libtorch thermochimica'

The JSON object will be a dictionary, where the keys are the capabilities (strings) and the values are bools (indicating if the capability is present - if the key is missing the capability is assumed to be not present) or numbers or strings. We will support a negation operator to enable tests explicitly if a capability is not present (useful for error checking)

capabilities = '!thermochimica'

There should also be a syntax for equality and inequality checks for non-bool capability values (again, missing keys will automatically return false).

To properly support version numbers we should detect . in strings and perform a left to right comparison on the splits.

Impact

Generalized and more robust capability checking.

GiudGiud commented 8 months ago

pretty neat idea. It will clean up this proliferation of test syntax.

how do you plan to handle misspells so

capabilities = 'libtorc' which would have been caught with libtorc = true (would not have been caught with required_submodules)

is there going to be a fixed capability registry? or maybe we should rely on the detection of unused inputs in tests from #24712