Propose a space delimited UCI

[timcera]

The import_uci function takes a HSPF UCI file and imports the tables into the HDF5 file for hsp2 to run. Using this format and process though imposes the HSPF limits that don't exist in hsp2, the most constraining being the three digit limits on PERLND, IMPLND, and RCHRES identifiers. Proposing a space delimited UCI where an @ symbol is used to indicate a default.

HSPF UCI snippet:

  PWAT-PARM3
    <PLS >***
    # - #*** PETMAX PETMIN INFEXP INFILD DEEPFR BASETP AGWETP
    1   7                            1.6
    9            39     33    3.0    1.5
   10
  END PWAT-PARM3

Proposed space delimited UCI for hsp2 (note the ability to have very large PERLND ids):

  PWAT-PARM3
    <PLS >***
    # - #***     PETMAX PETMIN INFEXP INFILD DEEPFR BASETP AGWETP
    10100   10107   @   @     1.6    @    @   @
    90100   @    39     33    3.0    1.5  @  @  @
    9010     @    @    @    @    @    @   @
  END PWAT-PARM3

Maybe not need trailing "@" default symbols...

  PWAT-PARM3
    <PLS >***
    # - #***     PETMAX PETMIN INFEXP INFILD DEEPFR BASETP AGWETP
    10100   10107   @   @     1.6
    90100   @    39     33    3.0    1.5 
    9010
  END PWAT-PARM3

[timcera]

Thinking about this I think a better approach is to have a CSV file for each table because it would be easier to create with GIS or other external processes. The file name would be the name of the block. So something like:

PWAT-PARM3.csv

ID_FIRST,ID_LAST,PETMAX,PETMIN,INFEXP,INFILD,DEEPFR,BASETP,AGWETP
10100,10107, , ,1.6
90100, ,39,33,3.0,1.5

Empty fields would be filled with the default for that parameter. Thinking that the header row would be convenience for the user and the order of the parameters would have to match the table in the UCI.

This would work very well with parameter estimation with PEST.

Then have a new command... hsp2 import_csvs *.csv new_model.h5

[rburghol]

The folks at the ches bay program have long used CSVs for storing 100s of subwatersheds, similar to what you propose here. I generally like the CSV from the standpoint of easy conversion to a queryable table, however, I'm also quite partial to the use of JSON, with individual sub-watersheds havin their own self-contained JSON description -- it's highly readable, and IMO intuitive to describe new things, and extensible by definition.

• Example of CBP (SEDTRN): https://github.com/HARPgroup/cbp_wsm/blob/master/input/param/river/p532cal/SEDTRN.csv
• Example of JSON (hsp2 ops model): https://github.com/HARPgroup/HSPsquared/blob/develop-equ-advanced/tests/testcbp/HSP2results/JL1_6562_6560.json.confluence

[timcera]

My immediate reaction to JSON is, well, ... let's just say I don't like it.

"These new-fangled phones. How do you type a phone number with one button?"

That being said, maybe it's a good idea. The advantage to CSVs is that you can take advantage of spreadsheets, GIS, Pandas, R (which I admit all might support JSON in some way or another), however you can tap into the familiarity that people have with CSVs. JSON in my mind is similar to XML which is appropriate for some things, however tiresome and tedious to read or edit.

Maybe could support both, though. Just have to come up with the JSON standard. Having all parameters for a particular RCHRES in a single JSON file is possibly interesting, but is a different way of thinking about things compared to a UCI (obviously).

The standard for the imagined CSVs is that they would follow the HSPF tables one-to-one, just comma delimited rather than fixed width. Looking at the link you sent for SEDTRN - it is one HUGE table. I am not imagining anything like that.

[rburghol]

Having had the same reaction to JSON as @timcera initially, I can say happily that while XML has been nothing but a headache to me, JSON is a much more friendly platform. But, to your point, the adoption would require agreeing on an approach. That said, much of the standard guts of JSON just work with numeric and character attributes, so, there is less to do than one might think. And we would be free to have a single RCHRES in a single file, or multiple RCHRESs, separately in a single file, or, even having one block of JSON for one UCI block (though that might undermine the readability benefit of JSON).

The idea on the one CSV to one table is straight-forward of course, I just included that CBP example for reference, although, I think their lesson is that having a single file for every single UCI block can get tedious.

Agree wholeheartedly that this doesn't need to be a one approach only type of endeavor.

[mishranurag]

I am also partial towards JSON. It is very easy to produce and read from different programs.

It is extensible and can be as simple as we like or as complex as we like. ~A

On Mon, Jul 29, 2024, 8:13 AM rburghol @.***> wrote:

Having had the same reaction to JSON as @timcera https://github.com/timcera initially, I can say happily that while XML has been nothing but a headache to me, JSON is a much more friendly platform. But, to your point, the adoption would require agreeing on an approach. That said, much of the standard guts of JSON just work with numeric and character attributes, so, there is less to do than one might think. And we would be free to have a single RCHRES in a single file, or multiple RCHRESs, separately in a single file, or, even having one block of JSON for one UCI block (though that might undermine the readability benefit of JSON).

The idea on the one CSV to one table is straight-forward of course, I just included that CBP example for reference, although, I think their lesson is that having a single file for every single UCI block can get tedious.

Agree wholeheartedly that this doesn't need to be a one approach only type of endeavor.

— Reply to this email directly, view it on GitHub https://github.com/respec/HSPsquared/issues/171#issuecomment-2256209179, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADTCNZ3E5LNW5UB5W62NPSTZOZLZVAVCNFSM6AAAAABKX5CJY2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENJWGIYDSMJXHE . You are receiving this because you are subscribed to this thread.Message ID: @.***>