Methods for configuring drivers

[inducer]

I saw #1 fly by, luckily I read the description instead of the title. I think using YAML as a configuration file format is not a great idea. You can use plain Python in much the same capacity:

def read_py_config(filename):
    with open(filename) as inf:
        file_contents = inf.read()

    config_dict = {}
    exec(compile(file_contents, filename, "exec"), config_dict)
    return config_dict

IMO, this is much more versatile because

• you can import stuff (including from other config files!)
• If you stick your configuration data into a class, you can use inheritance to override selectively without having to copy the whole thing.

cc @mtcam

[MTCam]

IMO, this is much more versatile because (...)

I agree completely. It sounds like one could make something like my_parameters.py just be a dictionary and then just do something like:

import my_parameters
dt = my_parameters["dt"]

... and let the Python do the whole sort it out on the filesystem thing. Would this work?

[inducer]

... and let the Python do the whole sort it out on the filesystem thing.

Yep, that's even better.

Would this work?

import my_parameters as par
dt = my_parameters.dt

works.

I think using a class instead of a file might be helpful still, given that you can do things like this:

class BaseParameters:
    dt = 1.23
    order = 5
    mesh_char_length = 0.1

class DerivedParameters(BaseParameters):
    order = 7

[MTCam]

class BaseParameters:
    order = 5
class DerivedParameters(BaseParameters):
    order = 7

Yup, I'm sold on this. It's so simple we don't need to provide a built-in construct for it. It would be nice to agree on a "standard" name for this parameter object, but I think it would be fine to just start using this in practice here and in the mirgecom examples, for example (heh), just let it become part of the dna so-to-say.

However, there is much utility in having a driver file that differs only by its input parameters among a set of many runs. That is largely where the pressure to use a file comes from I think. When folks like @anderson2981 design a run, they'll have many runs going at once, and having a separate (almost entirely duplicate) driver implementation for each of those is cumbersome. 🤔

[inducer]

When folks like @anderson2981 design a run, they'll have many runs going at once, and having a separate (almost entirely duplicate) driver implementation for each of those is cumbersome.

The parameters class could be imported:

from my_parameters import RunParameters2021_05_24

That way, the driver can stay the same.

Then the class docstring could even be used to summarize what the run was about. It can also contain code (for ICs?) if desired.

I think an important additonal trick might be to not change those parameter classes once they've been used, for archival. (and then delete them/roll up the changes from a long chain of inheritance into a new class, once no longer needed)

[MTCam]

The parameters class could be imported:

from my_parameters import RunParameters2021_05_24

(...) class docstring could even be used to summarize what the run was about. It can also contain code (for ICs?) if desired.

I'm picking up what you're putting down here. Your groove is correct.

I think an important additonal trick might be to not change those parameter classes once they've been used, for archival. (and then delete them/roll up the changes from a long chain of inheritance into a new class, once no longer needed)

I agree, this is definitely a good practice.

Often doing it right can get tricky, however. The parameters are likely to change over the course of the sim, and maintaining a record of those changes would be nice. One possible mechanism would just be to automatically save the parameter file with a timestamp on startup (including restarts). A cool side-effect is that one could tell at a glance when the sim was restarted by looking at the file listing in the output directory, where for each restart there'd be an extra file with containing the parameters used at that restart. (edit: maybe even go nuts and save the driver at every start/restart!)