Enhancements to Add or Modify Data Tool and Workflow

[nickrsan]

Original report by Nicholas Santos (Bitbucket: nickrsan, GitHub: nickrsan).

---

Add or Modify data should work by default against PISCES datasets (such as the QC datasets). When data is removed, it ends up being removed from all datasets, which makes updates to the data problematic because prior work is destroyed or changed. Instead, additions and removals should, by default only operate as:

1. Addition: Add new records, add to selected dataset
2. Transfer: If the records are in an existing dataset that isn't the dataset being worked on, copy them, then perform the transfer on the copy and add those records to the current dataset. If they are only in the current dataset being worked on, perform the transfer on them. If they are in no dataset, transfer them, then add them to the current dataset being worked on.
3. Removals: Should only remove records from the current dataset being worked on.

Optionally, some of these should be global, behaving as they currently do, allowing for transfers and removals for data management purposes.

This might require a rework of the workflow. The un-qced maps would no longer be useful because removals won't affect them. Instead, as we start an editing process, we'd need to copy all records from an existing dataset into a new one, and then import any new records into that, and edit from there. Creates other potential issues in that we then mix QCed and non-QCed data, but we can find that data based on its imported IDs. Could also have another tool in the toolbox that can pull in ranges based on the standard parameters.

Once all of this is done, we need to make a few modifications to data:

From peter:

"Thanks, Megan. Here are some changes I can see needed on the three maps you just sent.

PK Brook Lamprey

Please include the Lost River and Clear Lake Reservoir watersheds in the historic native range.

Label Goose Lake

Eagle Lake tui chub

Remove Willow creek from map as translocated site. I am not certain of the origins of the chubs in the creek.

Eagle Lake rainbow

For Eagle Lake itself include just the HUCs for the lake, not for South end of lake.

If Pine Creek can be labeled that would be good.

Eliminate the historic range only part of the map, on E side of Pine Creek watershed. Not sure what it refers to.

In answer to your questions:

For the counts for the H-12 watersheds, what level do you want me to search for these? Historical Range? Observed? Do you need the names of the H-12s? Are you interested in all the H-12? Or just the ones in Stanislaus, Lassen, Plumas, Tahoe, and El Dorado?

For HUC 12s use entire historic range fro total HUCs and then any category that occurs on USFS land, by forest. I need HUC 12 numbers for all national forests in which a species occurs. PK brook lamprey occurs in Modoc, Lassen, and Klamath forests.

Hope this is understandable."

[nickrsan]

Original comment by Nicholas Santos (Bitbucket: nickrsan, GitHub: nickrsan).

---

Another modification - PET01 - Pacific Lamprey is extirpated from Big Sur south.

[nickrsan]

Original comment by Nicholas Santos (Bitbucket: nickrsan, GitHub: nickrsan).

---

Process-wise, probably should temporarily define a variable for the dataset that's being edited and make a new dataset, copying any observation that's in either 5 or 15 into that new dataset, then use that as hq_collections in the future. Then make add or modify just remove from that defined variable's dataset - could be an option for the tool that is hidden by default.

The sticky part is detecting behavior for working with collections vs raw data - sometimes we do want to toss data, and are editing data that's not in any dataset at all. We may want to have a toggle for editing raw data vs datasets, and for adding any new records to the qc dataset too.