OQ/Hamlet fails when job.ini is a dict

[cossatot]

Hamlet uses the oq-engine to parse the source model based on the ssmLT.xml and return a list of ruptures. For a while, this worked when there wasn't a defined job.ini file, but Hamlet would construct a dictionary with the relevant parameters to parse the SSM.

However, this now fails with the following error (omitting the long stack of Hamlet functions):

  File "/Users/itchy/src/gem/hamlet/openquake/hme/utils/io/source_reader.py", line 13, in csm_from_job_ini
    calc = run_calc(
  File "/Users/itchy/src/gem/oq-engine/openquake/calculators/base.py", line 1382, in run_calc
    with logs.init("job", job_ini) as log:
  File "/Users/itchy/src/gem/oq-engine/openquake/commonlib/logs.py", line 303, in init
    return LogContext(job_ini, calc_id, log_level, log_file,
  File "/Users/itchy/src/gem/oq-engine/openquake/commonlib/logs.py", line 203, in __init__
    self.tag = tag or get_tag(self.params['inputs']['job_ini'])
KeyError: 'job_ini'

This is really useful during model development when we aren't to the point of defining a ground motion model logic tree (much less having sites, etc.) but just want to make sure the earthquake rupture forecast component is calibrated.

As a side note, I don't think that Hamlet should be incorporated into the oq-engine but I do wonder if the oq-engine's CI could run Hamlet's test suite to identify when these issues may occur.

[cossatot]

If it's at all relevant, here is the function that Hamlet uses to generate a job.ini dict:

def make_job_ini(
    base_dir: str,
    lt_file: str = "ssmLT.xml",
    gmm_lt_file: str = "gmmLT.xml",
    description: Optional[str] = None,
):
    ssm_lt_path = os.path.join(base_dir, lt_file)
    gmm_lt_path = os.path.join(base_dir, gmm_lt_file)
    job_ini_params = {
        "general": {
            "calculation_mode": "preclassical",
            "description": description,
        },
        "calculation": {
            "rupture_mesh_spacing": 2.0,
            "area_source_discretization": 15.0,
            "width_of_mfd_bin": 0.1,  # typically smaller than from cfg; use cfg?
            "maximum_distance": 200,
            "investigation_time": 1.0,
            "source_model_logic_tree": ssm_lt_path,
            "gsim_logic_tree": gmm_lt_path,
        },
        "site_params": {
            "reference_vs30_type": "measured",
            "reference_vs30_value": 800.0,
            "reference_depth_to_1pt0km_per_sec": 30.0,
        },
    }

    job_ini_params_flat = {k: v for k, v in job_ini_params["general"].items()}
    job_ini_params_flat.update(job_ini_params["calculation"])
    job_ini_params_flat.update(job_ini_params["site_params"])

    job_ini_params_flat = {k: str(v) for k, v in job_ini_params_flat.items()}
    job_ini_params_flat["inputs"] = {
        "source_model_logic_tree": str(ssm_lt_path)
    }

    return job_ini_params_flat

[micheles]

Being able to pass a dictionary instead of a job.ini file is an essential feature that I want to support forever, not only for Hamlet-sake. I am very surprised that it is broken now. Can you point me to the Hamlet CI? Unfortunately I do not get notifications when it breaks, that is the issue.

[micheles]

BTW, your function make_job_ini looks wrong, you are missing the job_ini key which is mandatory. Just write

    job_ini_params_flat["inputs"] = {
        "job_ini": "<in-memory>",
        "source_model_logic_tree": ssm_lt_path}

[cossatot]

Thanks Michele, that resolved the issue.

The Hamlet CI is found here: https://github.com/GEMScienceTools/hamlet/blob/master/.github/workflows/test_page.yaml

It doesn't seem to be triggered anymore--I'm not sure why, the file hasn't changed in a couple years and I didn't notice since I test locally before pushing typically and don't get email updates for successful test runs. Perhaps @vot4anto knows, since he made that file apparently?

[micheles]

It is a "feature" of the actions on GitHub. If nothing changes, after a period (3 months, I believe) the runner is automatically disabled and we cannot stop that from happening except by remembering to change something in the repository every 3 months :-(