Field Slip Forms

[mo-nathan]

Allow creation of forms for Field Slip data. These forms would be associated with a Project and would be used on the create/edit page for an associated Field Slip. Should support strings and checkboxes (habitat).

[mo-nathan]

I've been thinking about this more and I think this is important for the recorder role at mushroom events. The recorder should be able to scan the QR code and immediately go to a page that let's them transcribe any data that's on the physical field slips. This is actually kind of different from the observation per se. You still want to link to an existing observation and ideally the data is the same, but I can see them being out of sync due to human error.

[mo-nathan]

I think project admins for a field slip should go to the edit field slip page which should have all the fields from a field slip. It would be cool if it was a show & edit page. Not sure those are still fashionable.

[mo-nathan]

Planning on reusing the concept of a notes template from users/observations only for projects/field slips. The template would by default have the fields for the current field slip. By default it would sync with the notes for the associated observation. The edit form page should have a way to add images to the associated observation and create the observation if it doesn't already exist. Should completing the field slip form create the observation? Might not have any images yet. Pretty sure the answer is yes. Would want to allow the collector to create the observation/add to the observation. Should provide a way to pull data from another MO (or iNat) obsie. Admin or field slip owner can move the field slip to point at a later obsie.

[mo-nathan]

Plan:

• [x] Update field slip form with relatively easy fields
• [x] Add support for structured Notes
• [x] Add to field slip show page and get data from associated observation
• [x] Update form to use the same data
• [x] Save data to an existing observation
• [x] Save data to a new observation
• [x] Propose Field_Slip_ID if observation doesn't currently have a name
• [x] Update field slip show page to use standard edit icon and other cleanup ? Add checkbox support for Notes ? Enable changing the notes template per project

[mo-nathan]

Having looked a bit at the idea of using notes templates as the primary approach for field slips, I don't think it makes sense outside of the "Notes" field. All the other fields have corresponding MO objects (users, locations, names) that would be reasonable to optionally associate with the field slip data if it exists. I'm thinking through what sort of autocomplete options to provide. Does it make sense to involve the create location and name workflow if they enter values that aren't already in MO? I think we have to allow that for people since we cannot assume that the collector or identifier have MO accounts. It would also be helpful for the identifier initials and the locations to be easy to find, but maybe that's a follow on feature.

[mo-nathan]

I'm realizing that it doesn't make sense for the Field Slip and the Observation to be out of sync. The only situation I can think of where the information on the Field Slip might be different form the Observation and it wouldn't be possible to essentially force this information is if the creator of the observation doesn't trust the project and the recorder (a project admin) wants to override the observation data due to the information on the Field Slip. In any other case, the recorder can fix the information, or create the observation. In the case where they create the observation, the "Collector" field and the "Notes" can be stored directly on the observation. The "ID" can come in as a naming for the observation. The "ID By" info can be stored on the observation. Ideally if the "ID By" is a known MO user they would own the naming, but I think that ok given that it will only be initials. Also, hopefully over time the folks doing the IDs would start doing it directly using the QR code and their phone.

[mo-nathan]

When looking to populate the form fields I've been looking at the usage of notes templates for existing observations. There is a fair amount of usage (1000s of observations) and it's is common for there to be data that corresponds to data for the field slip forms. There are currently 126 distinct note template fields. The relevant ones for the field slip data relate to who the collector was (2552 cases), what the name/id was (2445 - dominated by the Ceskas), and who the identifier was (655). In all three cases there is more than one field name used and in many cases the field name is used by more than one user:

Collector:
:Collector or :collector, 560 cases, 5 users
:Collector's_Name or :Collector's_name, 1289 cases, 19 users
:Collector(s), 703 cases, 3 users

ID_By:
:Identifier, 354, 4 users
:Identified_by, 301, 1 user (Ceskas)

ID:
:Preliminary_Identification, 452, 1 user (Joe)
:Original_ID, 257, 1 user (Ceskas)
:Original_Herbarium_Label, 1599, 1 user (Ceskas)
:Accepted_ID, 8, 1 user (Ceskas)
:"Accepted_ID/Name", 129, 1 user (Ceskas)

Note that case sort matters in the code, but the MySQL LIKE matcher ignores case. I don't think there is any reason to actually treat case differences differently so I've lumped them together above.

So the core question is how should this data be used relative to the field slip data. It seems reasonable if someone is applying a field slip to an existing observation to look for matches that seem to be carrying relevant data. The big question is what to do if the user saves field slip data with changes. Should the existing field get overwritten? For observations without this data we need field names. My initial thought was "ID", "ID_By", and "Collector". However, given that "Collector" is already in use, maybe the field slip fields should have a distinct prefix (e.g., "Field_Slip_ID", "Field_Slip_ID_By", and "Field_Slip_Collector")? This could lead to some duplication of data, but I suspect that's the right approach vs. potentially unintentionally overwriting data.

[nimmolo]

Have not thought about this deeply enough to have a strong opinion, but didn't want to ignore. It seems to me on the merits of code simplicity that the field slip notes should maybe stomp the obs notes and we just maintain one record, but you've thought about this more than I. I like the idea of condensing "Collector's Name" in to "Collector" and generally forcing convention over configuration when useful.

[mo-nathan]

In the context of the PR for this work, #2160, I've realized that managing the inclusion of observation in project is a bit tricky. As a result, I've created the following writeup which I think covers the relevant cases.

How to Manage Project Inclusion for MO Observations

There are a lot of potential Project configurations. In this document I've considered the following properties, any of which may be true or false for a given Project.

• Allows anyone to join?
• Has a field slip prefix?
• Has date or location constraints?
• Already has any Observations?

I believe the relevant actions that might change Project inclusion for an Observation are:

• Edit an Observation
• Create an Observation without a Field Slip
• Scan a Field Slip and create a new Observation
• Scan a Field Slip and assign it to an existing Observation
• Edit a Field Slip

For any Projects, we also need to consider whether this User is a member, an admin, or not a member.

Edit an Observation

The Project check boxes should only list Projects the current User is a member (or admin) of. Initially they should reflect the Projects the Observation if currently part of. The User can then select any changes they want. For a given Project that the Observation is being added to, if the Project constraints are met or the User is an admin for that Project then the Observation is added. Otherwise the User is told to contact an admin for that Project (note there could be more than one). Admins are given a warning message about constraint violations, but the change is made.

Create an Observation without a Field Slip

Propose Projects the User has joined since they last created an Observation and which have Observations, as well as the Projects the User's most recent previous Observation is part of. On save, reject with a message any Projects that they are not admins for where there is a constraint violation. Warn admins before allowing constraint violations, but allow them to override the behavior.

Scanning a field slip code:

If a Field Slip object is already associated with a field slip code and it has a Project, then that is the "Project of the Field Slip". Otherwise, it is the most recently created Project whose field slip prefix matches the prefix of the field slip code. If no such Project exists, then there is no "Project for the Field Slip".

When a field slip code is scanned or otherwise entered into the system, if there is no associated Field Slip object, but there is an associated Project, that Project is open, and the current User is not a member, then that User is added to the associated Project as a member. This is regardless of whether a Field Slip object is actually created at that time.

Create a new Observation

Consider the Project of the Field Slip. If the User is a member, propose that Project. If not, provide a warning and do not propose that Project. Also consider any Project that would be proposed above in "Create an Observation without Field Slip". If the Project of the Field Slip exists and is proposed, then only propose other Projects that don't have Field Slip prefixes. Save behavior is the same as "Create an Observation without a Field Slip".

Assign to existing Observation

Consider the Project of the Field Slip. If all constraints are met or the User is an admin for that Project then add the selected Observation to that Project. Give admins a warning message if there are constraint violations.

Edit a Field Slip

There are two relevant actions here: 1) Changing the Project of a Field Slip and 2) Changing the Observation of a Field Slip. It is also possible to do both at the same time. Note that to edit a Field Slip the User must either be the owner of the Field Slip or an admin for the Project currently associated with the Field Slip. In addition Users can only select Projects that they are already a member (or admin) of.

As a result of the edit, the target Observation should get associated with the target Project. If before the edit, the Field Slip is associated with a different Observation, then the User should be given the option of whether to keep that Observation associated with the relevant Project or to remove that association. I tried to figure out if there are any permission issues here, but I think as long as the User has the ability to edit the Field Slip, then I think all the changes they can make are reasonable.

[JoeCohen]

Another action which can change Project inclusion for an Obs: The Project admin removes from the Project an Observation which violates Project Constraints.

[mo-nathan]

Fair, but an easy case. I'm mostly concerned about when an Observation gets added to a Project.

[JoeCohen]

Overall comments:

• The logic is complex. Is there a way to simplify it? Nothing jumps out at me.
• If I'm understanding this correctly:
  • If a user takes home Field Slips from a Project, and later scans a Field Slip and creates an Observation,
  • then the Observation is added to the Project (unless the User unchecks the suggested Project).

[mo-nathan]

Oops, good catch! The intent is for such Observations to be checked against the Project constraints as well on save and for them to be rejected if any constraints are violated (and the user isn't an admin). Updated the above comment.

[mo-nathan]

I think these are the gaps in the current behavior on the njw-field-slip-forms branch:

Edit Observation

• [x] ISSUE: Members can currently violate constraints Show an error if any project constraint is violated for a project that the user is not an admin and let the user fix the issue. The error should name all the project that meet these criteria. Give a warning if the above is not true and any project constraint is violated for a project that they are an admin for. This is essentially already working.

• [x] BUG: Leaving “Ignore Project Constraints” unchecked behaves the same as having it checked.

Create an Observation without a Field Slip

• [x] ISSUE: Selecting too many Projects. Create an observation as Darvin, uncheck all projects. Creating another observation still has “Field Slip Testing” and “North Falmouth Fungi” checked by default. Note that both of those are “open”.

• [x] ISSUE: I don’t think the code is actually checking what projects the user has joined recently. Digging into this, there is no great way to figure out what projects the user has joined recently. So I decided to instead reuse the projects of the most recent previous observation. If there isn't one, then use any project the user is a member of which are "current".

Scanning a field slip code to Create a new Observation

• [x] ISSUE: Selecting wrong projects. Compute Project of the Field Slip and apply appropriate logic. Don’t select by default other projects with prefixes, if the Project of the Field Slip is included.

Scanning a field slip code to Assign to existing Observation

• [x] Check project constraints and give warning if they are violated but don’t add unless the user is an admin.

Edit a Field Slip

• [x] ISSUE: The previously associated observation remains associated with the project even if this was the only FieldSlip making that association. Observations with no field slips pointing to a project should be removed and the user should be given a warning.