esm-tools / esm_tools

Simple Infrastructure for Earth System Simulations
https://esm-tools.github.io/
GNU General Public License v2.0
25 stars 12 forks source link

Feat/provenance #1146

Closed mandresm closed 3 weeks ago

mandresm commented 3 months ago

The Provenance feature PR!

Every value in config has now a provenance attribute. Many back-end features are added to provenance.

TODO

@pgierz, you can use this branch to add this functionality. I have merged to this branch every Nadine's or your branches about the provenance and it's still working with the pytests and checkruns.

mandresm commented 3 months ago

And here the script for ruamel.yaml comments on the right of the values:

import ruamel.yaml
from io import StringIO

yaml = ruamel.yaml.YAML()  # defaults to round-trip

inp = {
    "abc": ["a"],
    "xyz": {
        "a": 1,
        "b": 2,
        "c": 3,
        "d": 4,
        "e": 5,
        "f": 6,
    }
}

# Use StringIO as a stream for dumping the YAML content
stream = StringIO()
yaml.dump(inp, stream)

# Load the YAML string back into a Python object
stream.seek(0)
data = yaml.load(stream)

print(type(data))

# Modify the data and add comments
data['abc'].append('b')
data['abc'].yaml_add_eol_comment('comment 4', 1)  # takes column of comment 1
data['xyz'].yaml_add_eol_comment('comment 5', 'c')  # takes column of comment 2
data['xyz'].yaml_add_eol_comment('comment 6', 'e')  # takes column of comment 3
data['xyz'].yaml_add_eol_comment('comment 7\n\n# that\'s all folks', 'd', column=20)

# Dump the modified data back to YAML
stream = StringIO()
yaml.dump(data, stream)

modified_yaml_str = stream.getvalue()
print(modified_yaml_str)
mandresm commented 1 month ago

Hi @pgierz, this is ready to be reviewed. Actual tests are passing in levante, the cleanup is done, docstrings written...

mandresm commented 1 month ago

Provenance documentation here:

https://esm-tools.readthedocs.io/en/feat-provenance/yaml_hierarchy.html#how-can-i-know-where-a-parameter-is-defined

mandresm commented 3 weeks ago

approve-changes

mandresm commented 3 weeks ago

bump