Transport-for-the-North / caf.toolkit

Toolkit of transport planning and appraisal functionalities.
https://caftoolkit.readthedocs.io/en/stable/
Other
0 stars 2 forks source link

Add `write_metadata` function #124

Open MattBuckley-TfN opened 4 months ago

MattBuckley-TfN commented 4 months ago

Create new function in log_helpers for writing model metadata to a YAML file, should be given ToolDetails and any metadata arguments.

SystemInformation.load() should be used to output that information to the YAML file.

Example of the function signature below, where metadata keyword arguments can be anything which can output to a YAML file e.g. standard Python types including int, str, float, dict, list ...

def write_metadata(path: pathlib.Path, details: ToolDetails, **metadata: Any) -> None:

Example of the output file:

# Tool details provided by function caller
tool_details:
  name: caf.toolkit
  version: 0.1.0
  homepage: https://caftoolkit.readthedocs.io/en/stable/
  source_url: https://github.com/Transport-for-the-North/caf.toolkit

# System information automatically determined using SystemInformation.load
system_information:
  user: PC User Name
  pc_name: Name of PC running
  python_version: 3.12
  operating_system: Windows 10
  architecture: AMD64
  processor: Intel
  cpu_count: 4
  total_ram: 100

# All the keyword arguments passed to the function
metadata:
  key: value
  date: 2024-06-19
  project: Project Name
  owner: Owner's name
  stakeholders:
    - Stakeholder 1
    - Stakeholder 2
    - Stakeholder 3
  inputs_folder: Path/to/inputs
MattBuckley-TfN commented 2 months ago

130 is suggesting a very similar solution, albeit slightly more complex. This write_metadata function could provide an additional configs parameter which expected to be given a dictionary of BaseConfig instances and will write those out to the YAML file using the dictionary keys.

MattBuckley-TfN commented 2 months ago

Potentially could be implemented using a BaseConfig subclass to handle saving to a YAML file an example implementation is below.

class _MetaData(BaseConfig):
    tool_details: ToolDetails
    system_information: SystemInformation | None = None
    configs: dict[str, BaseConfig] | None = None
    metadata: dict[str, Any] | None = None

def write_metadata(
    path: pathlib.Path,
    details: ToolDetails,
    system_info: bool = True,
    configs: dict[str, BaseConfig] | None = None,
    save_kwargs: dict[str, Any] | None = None,
    **metadata: Any,
) -> None:
    if system_info:
        sys_info = SystemInformation.load()
    else:
        sys_info = None

    if save_kwargs is None:
        save_kwargs = {}

    data = _MetaData(
        tool_details=details, system_information=sys_info, configs=configs, metadata=metadata
    )
    data.save_yaml(path, **save_kwargs)