aekiss / run_summary

Summarise ACCESS-OM2 runs
Apache License 2.0
1 stars 1 forks source link

Namelists incorrectly serialised #24

Closed aidanheerdegen closed 1 year ago

aidanheerdegen commented 1 year ago

Namelists are serialised directly leading to non-standard yaml

e.g. Running 1deg_era5_ryf for a year and then

$ python run_summary.py -d /home/502/aph502/payu/access-om2/1deg_jra55_ryf  

produces output like so

  namelists:
    accessom2.nml: !!python/object/apply:f90nml.namelist.Namelist
      dictitems:
        ? !!python/object/new:f90nml.namelist.NmlKey
          args:
          - accessom2_nml
          state:
            _key: accessom2_nml
        : !!python/object/apply:f90nml.namelist.Namelist
          dictitems:
            ? !!python/object/new:f90nml.namelist.NmlKey
              args:
              - ice_ocean_timestep
              state:
                _key: ice_ocean_timestep
            : 5400
            ? !!python/object/new:f90nml.namelist.NmlKey
              args:
              - log_level
              state:
                _key: log_level
            : DEBUG
          state:
            _cogroups: {}
            _column_width: 72
            _default_start_index: null
            _end_comma: false
            _float_format: ''
            _indent: '    '
            _index_spacing: false
            _logical_repr:
              false: .false.
              true: .true.
            _newline: false
            _repeat_counter: false
            _split_strings: false
            _start_index: {}
            _uppercase: false

as the f90nm.namelist.Namelist is serialised naively.

Fix is to probably use the todict method when saving the parsed namelist data.

I tried this as a simple fix and get some weirdness, so will take another look tomorrow.

aidanheerdegen commented 1 year ago

Ah ha, following this https://stackoverflow.com/a/50181505 I added

yaml.add_representer(OrderedDict, lambda dumper, data: dumper.represent_mapping('tag:yaml.org,2002:map', data.items()))

to the top to get the OrderedDict serialising correctly. Now I get output like so:

  namelists:
    accessom2.nml:
      accessom2_nml:
        log_level: DEBUG
        ice_ocean_timestep: 5400
      date_manager_nml:
        forcing_start_date: '1900-01-01T00:00:00'
        forcing_end_date: '1901-01-01T00:00:00'
        restart_period:
        - 1
        - 0
        - 0
    atmosphere/atm.nml:
      atm_nml: {}
      runoff_nml:
        remap_weights_file: INPUT/rmp_jrar_to_cict_CONSERV.nc
    ice/cice_in.nml:
      setup_nml:
        days_per_year: 365
        dbug: false
        diag_file: ice_diag.d
        diag_type: file
        diagfreq: 960
        dt: 3600
        dump_last: true
        dumpfreq: y
        dumpfreq_n: 1
        hist_avg: true
        histfreq:
        - d
        - m
        - x
        - x
        - x
        histfreq_n:
        - 1
        - 1
        - 1
        - 1
        - 1
        history_chunksize_x: 180
        history_chunksize_y: 150
        history_deflate_level: 1
        history_dir: ./OUTPUT/
        history_file: iceh
        ice_ic: default
        incond_dir: ./OUTPUT/
        incond_file: iceh_ic
        istep0: 0
        latpnt:
        - 90.0
        - -65.0
        lonpnt:
        - 0.0
        - -45.0
        ndtd: 1
        npt: 35040.0
        pointer_file: ./RESTART/ice.restart_file
        print_global: false
        print_points: false
        restart: false
        restart_dir: ./RESTART/
        restart_ext: false
        restart_file: iced
        runtype: initial
        use_leap_years: false
        use_restart_time: true
        write_ic: false
        year_init: 1
      grid_nml:
        grid_file: RESTART/grid.nc
        grid_format: nc
        grid_type: tripole
        kcatbound: 0
        kmt_file: RESTART/kmt.nc
      domain_nml:
        distribution_type: cartesian
        distribution_wght: latitude
        ew_boundary_type: cyclic
        maskhalo_bound: true
        maskhalo_dyn: true
        maskhalo_remap: true
        nprocs: 24
        ns_boundary_type: tripole
        processor_shape: slenderX1
aekiss commented 1 year ago

Thanks @aidanheerdegen, happy to take a PR with this fix