Consider revising aggregation methods

[rabutler]

Right now, we combine the summary step and the comparison against a threshold, e.g., AnnMax, EOCYLTE, EOCYLT, etc. This results in having a lot of aggregation methods.

We could split this into two steps. Step 1, would be the aggregations, e.g., sum, max, min, raw. Step 1a would be either CY or WY. Step 2 would then be the summary, e.g., <, <=, >, >=, !=, ==, NA.

[rabutler]

May want to re-structure how the getData and aggregate data works. could:

• [x] 1. get the Data from rdf and turn it into a long table - identical to the table generated by RiverWare/RiverSMART (see #30 rw_rdf_to_tbl())

• [x] 2. then call aggregation methods after that. This ensures that the same aggregation code would work whether you are starting with rdf files or the csv file from RW

  • [x] First determine your time aggregation: pass function CY(), WY(), none, EOCY, EOWY, specific month, or custom function. Then you could write a custom function, e.g., summer() which filters only June, July, August.
    • [x] If tbl does not include year and month, will need to add them via add_ym_to_rdftbl()
  • [x] Then determine your summary function, min, max, sum, none
    • [x] how to specify na.rm = TRUE for mean, etc. (I think getDataForAllScens() treats NA as a 0. Check this. If it does always, then I guess we do that here too. Otherwise, maybe we edit the summary column to be able to accept multiple functions, e.g., sum(na_to_zero()), by separating with ;, or maybe piping them
    • getDataForAllScens() does treat NaNs as 0s. For now, we will offer two options that are decided per call to rwtbl_aggregate(): "0" or "error". The first treats NaNs as 0s, and the second as errors.
  • [x] Then evaluate comparison to threshold, or scale the data. If evaluating against threshold, take in "<", "<=", ">", ">=", "!=", "=="
  • [x] Then assign new Variable name, but store ObjectSlot as attribute? (or keep it)
    • in a 1 million row tbl from getDataForAllScens(), adding a second character column adds 25% onto the size of the tbl; same is true for one with ~500,000 rows. So I think we should just store the slot agg matrix as an attribute in the tbl; then we can read how the variable was created if we want to.
  • [x] Need to be able to handle the "all" keyword in the rwd_agg

• [x] 3. See #50 for all of the different attributes we need to add to the rwtblsmmry

  • [x] store the slot agg list as an attribute, or some way to keep track of how slots were summarized
  • [x] maintain original folder name, and scenario name
  • [x] maintain original slot name, and variable name

• [x] 4. New version of getDataForAllScens() that calls rwtbl_aggregate()

  • [x] Need to implement all the options we have in getDataForAllScens()
  • [x] make sure it returns all the attributes in step 3
  • [x] tests that compare getDataForAllScens() to rw_scen_aggregate()
  • [x] examples in documentation; also for rwtbl_aggregate()

---

also could apply to the netcdf, eventually (#57)

[rabutler]

• need to figure out how this would interplay with the xts data and aggregation code.
  • maybe not at all, maybe that's completely seperate?
  • or maybe the table is turned into an xts object to allow easier date filtering?

[rabutler]

Might not need to do #14 if we do this first

[rabutler]

tests

• [x] check that for a single scenario of data the new aggregation methods work for csv or rdf input

will check csv in #56