ArctosDB / documentation-wiki

Arctos Documentation and How-To Guides
https://handbook.arctosdb.org
GNU General Public License v3.0
13 stars 13 forks source link

Entering specimen records in stages #134

Open Jegelewicz opened 5 years ago

Jegelewicz commented 5 years ago

Issue Documentation is http://handbook.arctosdb.org/how_to/How-to-Use-Issues-in-Arctos.html

Is your feature request related to a problem? Please describe. Due to the way data is accumulated in the field for some of our collections, locality information is separate from specimen part information. Some of the parts are tissues that are frozen in the field and end up in a freezer with no data, just a barcode. This makes it difficult to make use of the tissues.

Describe the solution you'd like We would like to be able to enter a specimen record without any events. In order to get tissues cataloged quickly, we would like to be able to enter specimen records that only include identification, collector, and part information. Events would be added later using the Specimen Event Bulkload Tool. (We would also like a "low quality data" report that would list the records without events so that these could be noted and followed up on.)

Describe alternatives you've considered The standard practice has been to wait to catalog anything until all of the data is compiled and the specimen preparation has been completed (which may be years after collection). Recently an enterprising student has entered "skeletal" records that include a generic locality with the idea that the actual locality will be added later using the Specimen Event Bulkload Tool. The problem with this solution is that this means each of these specimens will now include two events that create occurrences at GBIF and iDigBio and manually "unaccepting" the event with the generic locality will be too time consuming. We have considered the possibility of a "make existing event(s) unaccepted" option as part of the Specimen Event Bulkload Tool, but we would prefer in this case not to have the generic event appear at all.

Additional context We realize that workflows may be the issue here, but at this time we are unable to change the way field/prep work is completed by the research teams and we still need to properly manage the tissues which are in our freezers. Our enterprising student is trying to write a protocol so that those who come after her can follow it and make the process run smoothly.

Priority Please assign a priority-label.

dustymc commented 5 years ago

Dependencies aside, most anything that's "required" in the specimen bulkloader is because someone's asked for it to be. We can't create locality data without specimen_event_type (a NOT NULL field in a 'root' table, from that perspective), but we can certainly create a specimen record which contains no locality information.

I agree with your approach: get what you have saved with everything else when you have it. Doing that is likely to somewhat complicate the specimen data entry screen, and might require some more training ("you CAN leave that NULL, but don't...").

I don't see any insurmountable technical hurdles; this looks like a social issue.

Failing all of that, the loader/unloader approach seems to be working well for other things and I think it would be fairly straightforward to introduce a specimen event de-bulkloader. Who knows how that might ultimately work, but using named Events is very likely to make it easier. I suggest you create a "delete me" event for this until/unless the core issue is resolved.

Jegelewicz commented 5 years ago

the loader/unloader approach seems to be working well for other things and I think it would be fairly straightforward to introduce a specimen event de-bulkloader. Who knows how that might ultimately work, but using named Events is very likely to make it easier. I suggest you create a "delete me" event for this until/unless the core issue is resolved.

I like this idea better. Rather than a named event, how about if we just have a new verification status:

"placeholder" = The place and time information as entered are incomplete, this event will be removed when complete information is available for entry.

That makes it easy (I think) to find all of the incomplete events and get the appropriate information for a de-bulkloader. Also, I suggest that all events with "placeholder" status NOT be sent to aggregators (treat them as "unaccepted" for that purpose).

dustymc commented 5 years ago

placeholder

I'm not sure I'm morally opposed or anything, but it seems like a messy way of getting to the same place. 'Build a de-loader file for all specimen-events using {various stuff from event/locality/whatever}' seems a possibility, for example.

Jegelewicz commented 5 years ago

So maybe with a specific locality of "delete when locality data is entered" or something like that?

The thing is, we won't want to delete them all at any given time, just the ones where the ACTUAL locality has been entered...

dustymc commented 5 years ago

specific locality of "delete when locality data is entered"

100% gonna get a few "delete after locality data is entered" that you'll never find....

just the ones where the ACTUAL locality has been entered...

SELECT 
   whatever the bulkloader thingee needs 
FROM
    all_events
WHERE 
   one event uses some sort of controlled data so you can actually find it AND
   some other event exists or has coordinates or whatever

I think that works about the same no matter what's used as the temp-marker, my suggestion of named events is just unobtrusive (eg, not seen by everyone who ever tries to do anything with specimen events) and (unlike eg, spec locality) is controlled/predictable so it's easy to find when you need it.

That could run as part of specimenresults (if we can somehow figure out how to deal with multiple events, which seems unlikely given the tabular format), as part of the de-loader (doesn't seem unreasonable), or just as SQL that could be adjusted and reused as necessary.

campmlc commented 4 years ago

So I have this set of records that will need to have the existing event deleted or updated once the locality info is available. What would be the procedure? https://arctos.database.museum/saved/MSB%20delete%20no%20specific%20locality%20spec%20event%20remarks

dustymc commented 4 years ago

deleted

File a new issue requesting a specimen event de-bulkloader. I'll need details regarding how you'd like to find the specimen-event that needs removed.

or updated

There are lots of tools in the manage dropdown

campmlc commented 4 years ago

Does the update specimen event tool in Manage dropdown replace the existing event, or add new one?

dustymc commented 4 years ago

I believe it can do both if the existing data allow it.

campmlc commented 4 years ago

If I bulkload new events to a set of records, is there an option to delete existing event? This would be a bulkload where each record may have a different event.

On Wed, Oct 2, 2019, 3:26 PM dustymc notifications@github.com wrote:

I believe it can do both if the existing data allow it.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/ArctosDB/arctos/issues/2054?email_source=notifications&email_token=ADQ7JBDBBUQZCEXHRZ6GE7DQMTYW7A5CNFSM4HH7EB7KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAF4PWQ#issuecomment-537642970, or mute the thread https://github.com/notifications/unsubscribe-auth/ADQ7JBDI65HOYHFZEXUAHSDQMTYW7ANCNFSM4HH7EB7A .

Jegelewicz commented 4 years ago

Needs documentation - a really good How To.....

Jegelewicz commented 3 years ago

https://github.com/ArctosDB/documentation-wiki/blob/gh-pages/_how_to/How-to-Bulkload-Parts.markdown

Will be part of this