respec / HSPsquared

Hydrologic Simulation Program Python (HSPsquared)
GNU Affero General Public License v3.0
43 stars 17 forks source link

Handle irregular time series input #51

Open aufdenkampe opened 3 years ago

aufdenkampe commented 3 years ago

Presently, HSP2 can not handle irregular time series as inputs.

Although irregular time series inputs are not common for HSPF, @bcous has found a historical set of WDM files where the input time series started at 1 hour intervals and then switched to 15 minute intervals. HSPF resamples all inputs to the model time step immediately prior to a run, so everything works fine in HSPF.

With the recent successful Rewrite readWDM.py to read by data group & block #21, we can properly read these and all other tested WDM. However, as @ptomasula commented (https://github.com/LimnoTech/HSPsquared/issues/21#issuecomment-827178993), running HSP2 on those inputs will throw an error if tsfreq == None:

~/Documents/Python/limno.HSPsquared/HSP2/utilities.py in transform(ts, name, how, siminfo)
     78         pass
     79     elif tsfreq == None:     # Sparse time base, frequency not defined
---> 80         ts = ts.reindex(siminfo['tbase']).ffill().bfill()
     81     elif how == 'SAME':
     82         ts = ts.resample(freq).ffill()  # tsfreq >= freq assumed, or bad user choice

KeyError: 'tbase'

We have several options to fix this:

  1. Drop higher frequency data
  2. Fill entire time series to highest frequency, but how?
    • Fill options: NaN (or -999.0), previous value, interpolated value
  3. Split into two time series
  4. Modify HSP2.utilties.py code to handle it

We'll do option 1 in the short term (probably "manually"), but option 4 is probably the best long term fix.

This issue will track progress on option 4.