A new system to convert between UCI, CSV, JSON, and HDF5... - Githubissues

respec / HSPsquared

Hydrologic Simulation Program Python (HSPsquared)

GNU Affero General Public License v3.0

44 stars 17 forks source link

A new system to convert between UCI, CSV, JSON, and HDF5... #175

Open timcera opened 2 months ago

timcera commented 2 months ago

Working on a a new system that will convert between several representations and wanted to announce so that if someone is working on the UCI parser we could work together.

What I have so far:

New UCI parser that supports the Parameter ESTimation supplemental file which would close #172
Creates a dictionary of pandas DataFrames that can be written to HDF5, CSV, or JSON files which would close #171. The JSON files are direct representations of the HSPF tables, so not exactly what was discussed in #171, however the table based JSON is a one-line addition. Thought could pursue other JSON views later.

Current problem children:

SPEC-ACTIONS table is complex and different from all other tables. Leaving until last. Need to work with @rburghol to make sure get in place what he needs.
Using src/hsp2/hsp2tools/data/ParseTable.csv to shift between the different representations which has worked out really well. Right at the moment though I am not creating the table names for HDF5 correctly. I think a minor change either to the code or to ParseTable.csv will get that fixed.

timcera commented 2 months ago

Just looking at other issues and see @rburghol work on #90, #99, #105, #106, and #107. Quite a bit to take in.

With my new parser I can easily create UVQUAN and UVNAME pandas dataframes which can then be converted to CSV, HDF5, and JSON, however the conditional lines are a real problem to represent in a pandas dataframe and don't know what is needed.