Reverse inclusion of old AquaSat data, default to "strict" scripts, draft field tiering

AquaSat / AquaMatch_harmonize_WQP

https://aquasat.github.io/AquaMatch_harmonize_WQP/

MIT License

1 stars 3 forks source link

Reverse inclusion of old AquaSat data, default to "strict" scripts, draft field tiering #33

Closed mbrousil closed 1 year ago

mbrousil commented 1 year ago

This PR:

Reverses my previous work on including the old raw AquaSat data in the AquaMatch product. WQP is dynamic so the data will change over time. Mostly affects 2_download.R
Removes references to "strict" data handling, which was a holdover from the light partitioning project era. "Strict" methods remain the default and will likely be changed in the next few days. Variables have been renamed for this change and some scripts deleted.
Drafts a field tiering method/function following discussions with @matthewross07 and @mishafredmeyer (script 3_harmonize/src/assign_field_tier.R). A question here for Matt and Michael, do we consider Filterable or Supernate values for ResultSampleFractionText to be "filtered"? See line 19 of the script for context

Note that I'll likely be editing parameter harmonization scripts a lot this week so they will be changing, generally speaking.

matthewross07 commented 1 year ago

I see this got merged, which I assume was just you @mbrousil trying to keep making progress on the main branch? ResultsSampleFractionText is a confusing column, but Filterable, should be considered filtered and I guess so should supernate? It's possible we will want to pass this column on this time around, since their is some ambiguity here.

steeleb commented 1 year ago

I see this got merged, which I assume was just you @mbrousil trying to keep making progress on the main branch? ResultsSampleFractionText is a confusing column, but Filterable, should be considered filtered and I guess so should supernate? It's possible we will want to pass this column on this time around, since their is some ambiguity here.

Side note: I also wonder if this (the filtering flag) skews away from field tiering and towards method tiering.

mbrousil commented 1 year ago

Whoops my bad. I can revert if you'd prefer

matthewross07 commented 1 year ago

No not a big deal, more just making sure you still needed input. I do think we can all help each other if we need things to merge quickly we can let each other know though? @steeleb maybe something we can add to best practices? Like I get it that sometimes a merge just needs to happen within XX hours.

Just keep on keeping on @mbrousil . I do have a talk tomorrow on this so if you have a table and/or an updated filtering/tiering figure output by tomorrow AM, that would be great. Otherwise I can use some of the older output.

steeleb commented 1 year ago

@steeleb maybe something we can add to best practices?

Agreed! I'm going to add it as an issue for now.