Conversion of data frames in the DA forcing module into BMI compliant arrays and reassembly in troute: wrote routines for all dataframe import into troute, and applied it to most dataframes
Additions
Modules with auxiliary functions to help convert dataframes to/from arrays:
-- bmi_df2array: Functions to disassemble dataframes into constituent numpy arrays (or individual variables) of either integer or float types. This includes converting any string-based dataframe entries (such as gauge IDs) to arrays based on ASCII-encoding, wherein one string array-like object gets converted into two 1D numpy arrays: a series of ASCIIs in one int array and an array comprised of the length of each string entry. Datetime entries are converted to seconds relative to a troute-internal time reference (which is, a priori, arbitrary).
-- bmi_array2df: The reverse of the latter. If applicable (for string-like entries such as gauge IDs), first reassembling a list of strings from the two constituent files (ASCII encoded array and ASCII string length array); or, for time arrays, reassembling lists of Datetime objects. Then, reassemble the applicable dataframes.
Removals
On the t-route side, the dataframe import option is kept as a debugging feature, which can be enables through the flag "legacy_bmi_df" in DataAssimilation. This shall be removed in the near future.
Changes
Lion's share of the code changes are in model_DAforcing (dataframe -> array conversions), and DataAssimilation (array conversions -> dataframe. The latter is implemented in the {NudgingDA, PersistenceDA, RFCDA} child classes of AbstractDA.
In the (dataframe -> array) direction, the following dataframes were converted: usgs_df, reservoir_usgs_df, reservoir_usace_df, reservoir_rfc_dataframe, q0/t0, waterbody_df, lastobs
In the (array -> dataframe) direction, the following dataframes were implemented for bmi import into troute: usgs_df, reservoir_usgs_df, reservoir_usace_df, reservoir_rfc_dataframe. There was existing code for the last three (lite_restart and lastobs), which shall be harmonized with the current commit in the near future.
Testing
Should not interfere with normal operation. A full-featured BMI test case will be constructed.
Screenshots
Notes
-
Todos
Harmonizing lite_restart and lastobs code with the existing additions.
Checklist
[ ] PR has an informative and human-readable title
[ ] Changes are limited to a single goal (no scope creep)
[ ] Code can be automatically merged (no conflicts)
[ ] Code follows project standards (link if applicable)
[ ] Passes all existing automated tests
[ ] Any change in functionality is tested
[ ] New functions are documented (with a description, list of inputs, and expected output)
[ ] Placeholder code is flagged / future todos are captured in comments
[ ] Visually tested in supported browsers and devices (see checklist below :point_down:)
[ ] Project documentation has been updated (including the "Unreleased" section of the CHANGELOG)
[ ] Reviewers requested with the Reviewers tool :arrow_right:
Conversion of data frames in the DA forcing module into BMI compliant arrays and reassembly in troute: wrote routines for all dataframe import into troute, and applied it to most dataframes
Additions
-- bmi_df2array: Functions to disassemble dataframes into constituent numpy arrays (or individual variables) of either integer or float types. This includes converting any string-based dataframe entries (such as gauge IDs) to arrays based on ASCII-encoding, wherein one string array-like object gets converted into two 1D numpy arrays: a series of ASCIIs in one int array and an array comprised of the length of each string entry. Datetime entries are converted to seconds relative to a troute-internal time reference (which is, a priori, arbitrary).
-- bmi_array2df: The reverse of the latter. If applicable (for string-like entries such as gauge IDs), first reassembling a list of strings from the two constituent files (ASCII encoded array and ASCII string length array); or, for time arrays, reassembling lists of Datetime objects. Then, reassemble the applicable dataframes.
Removals
Changes
Lion's share of the code changes are in model_DAforcing (dataframe -> array conversions), and DataAssimilation (array conversions -> dataframe. The latter is implemented in the {NudgingDA, PersistenceDA, RFCDA} child classes of AbstractDA.
In the (dataframe -> array) direction, the following dataframes were converted: usgs_df, reservoir_usgs_df, reservoir_usace_df, reservoir_rfc_dataframe, q0/t0, waterbody_df, lastobs
In the (array -> dataframe) direction, the following dataframes were implemented for bmi import into troute: usgs_df, reservoir_usgs_df, reservoir_usace_df, reservoir_rfc_dataframe. There was existing code for the last three (lite_restart and lastobs), which shall be harmonized with the current commit in the near future.
Testing
Screenshots
Notes
-
Todos
Checklist
Testing checklist
Target Environment support
Accessibility
Other