LSSTDESC / tables_io

A small package to provide tools to read / write and convert tabular data for DESC
MIT License
1 stars 1 forks source link

Add support for writing nested dictionaries to HDF5 #91

Closed drewoldag closed 7 months ago

drewoldag commented 7 months ago

Initial attempt to add support for nested dictionaries with numpy array values.

Problem & Solution Description (including issue #)

The goal is to be able to write out to an hdf5 file a structure that mirrors a nested dictionary where the leaf values are all numpy arrays. The current approach introduces a bit of recursion into the writeDictToHdf5 file. Still need to write a good handful of tests.

The specific use case here should support the following:

def write_dict(filename, ensemble_dict, **kwargs):
        # ensemble_dict has the form { <str> : <qp.Ensemble> }
        # output_tables has the form { <str> : <tabularized qp.Ensemble> }
        # where a tabularized qp.Ensemble is a dictionary of dicts with leaf values that are numpy arrays.

        output_tables = {}
        for key, val in ensemble_dict.items():
            # check that val is a qp.Ensemble
            if not isinstance(val, Ensemble):
                raise ValueError("All values in ensemble_dict must be qp.Ensemble")

            output_tables[key] = val.build_tables()
        io.writeDictsToHdf5(output_tables, filename, **kwargs)

Code Quality

codecov[bot] commented 7 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 100.00%. Comparing base (4dadddb) to head (8475829).

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #91 +/- ## ========================================= Coverage 100.00% 100.00% ========================================= Files 8 8 Lines 674 683 +9 ========================================= + Hits 674 683 +9 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.