ArctosDB / arctos

Arctos is a museum collections management system
https://arctos.database.museum
60 stars 13 forks source link

event terminology #4036

Closed dustymc closed 2 years ago

dustymc commented 2 years ago

But I think this has always been a problem what exactly does specimen/event remark mean? It has never been clear.

Anyway, change this to "Collection Object Remark" to match data entry

image

Change this to "Verbatim Event Details" to match catalog record (but see below)

image

Change this to "Verbatim Event Details" to match data entry

image

Change this to "Catalog Record Event Remark" on the catalog record

image

Change this to "Catalog Record Event Remark" in data entry

image

And also make appropriate changes to search. I can do all of this and let everyone see it in test.

Originally posted by @Jegelewicz in https://github.com/ArctosDB/arctos/issues/3691#issuecomment-951095270

dustymc commented 2 years ago

I greatly dislike "Verbatim Event Details" - that "node" is much more than that. The rest I could live with, I think, but it's extremely verbose.

We should also be looking at APIs - they will not share this terminology. (And the above doesn't cover all non-API usage).

Please do not make arbitrary changes; this needs planned.

what exactly does specimen/event remark mean

It is remarks regarding the intersection of specimens - "catalog records" - and the event stack.

arctosprod@arctos>> \d specimen_event
                                              Table "core.specimen_event"
        Column         |            Type             | Collation | Nullable |                  Default                  
-----------------------+-----------------------------+-----------+----------+-------------------------------------------
 specimen_event_id     | integer                     |           | not null | nextval('sq_specimen_event_id'::regclass)
 collection_object_id  | integer                     |           | not null | 
 collecting_event_id   | integer                     |           | not null | 
 assigned_by_agent_id  | integer                     |           | not null | 
 assigned_date         | timestamp without time zone |           | not null | 
 specimen_event_remark | character varying(4000)     |           |          | 
 specimen_event_type   | character varying(60)       |           | not null | 
 collecting_method     | character varying(4000)     |           |          | 
 collecting_source     | character varying(60)       |           |          | 
 verificationstatus    | character varying(60)       |           | not null | 
 habitat               | character varying(4000)     |           |          | 
 verified_by_agent_id  | integer                     |           |          | 
 verified_date         | character varying(30)       |           |          | 
Jegelewicz commented 2 years ago

OK - I have had a few nights of sleep to dream about this and here are some thoughts.

First - I think that if the data entry screen were structured in the same way the catalog record is, there would be less confusion. (and I know it sort-of is, but in a different order)

Beyond that, let's talk about the "event stack"

I'm going to start with the top of the stack - the part where we assign an event to a catalog record, currently known as the "specimen/event" (I have been trying to rid us of specimen)

Here is a comparison of all the ways we refer to sections of this part of the event stack:

Arctos Page/What it does Assigned Event Type Event Assigned By Event Assigned Date Event Verification Status Event Verified By Event Verification Date Event Assignment Remark Event Habitat Event Source Event Method
Edit catalog record Specimen/Event Type Event Determiner Determined Date Verification Status Verified By Verified Date Specimen/Event Remark Habitat Collecting Source Collecting Method
Catalog record Event Type Event Type: assigned by Event Date Event Verification: Event Verification: verified by Event Verification: on Event Details: remark Event Details: habitat  Event Details: Collecting Source Event Details: Collecting Method 
Data entry (old) Specimen/Event Type Event Determiner Detr. Date VerificationStatus N/A N/A Specimen/Event Remark Habitat Coll. Src. Coll. Meth.
Data entry (new) Event Type Event Determiner Det. Date VerificationStatus N/A N/A Event Remark Habitat Collecting Source Collecting Method
Bulkloader builder specimen_event_type event_assigned_by_agent event_assigned_date verificationstatus N/A N/A specimen_event_remark habitat collecting_source collecting_method
Bulkload specimen-events specimen_event_type assigned_by_agent assigned_date verificationstatus N/A N/A specimen_event_remark habitat collecting_source collecting_method

So I can see why there is confusion but before we start changing stuff, I think we need to take a hard look at what this information is meant to convey and make sure that the labels fit. I'd like to see this whole part of the stack better reflect what it is actually doing - recording information about who, when, and why the associated event/locality are assigned to the object(s) or observation(s) this catalog record describes. If that is a poor description of what is going on, please give me a better one!

FWIW documentation says this (and needs to get rid of "specimen")

Specimen Event Specimen Events link Specimens to Collecting Events.

A few other musings:

  1. It is unclear to me as to why "habitat" falls within this section of the stack and is not a "collecting event" attribute.
  2. I am also feeling ambivalent about the placement of "collecting method" and "collecting source", but perhaps that will become more clear when I take a look at the "collecting event" part of the stack....
dustymc commented 2 years ago

data entry screen were structured in the same way the catalog record is

It won't be - one has a fixed structure (with some "extras"), the other is purely data driven and has no limitations.

different order

The entry screen has no order at all (and maybe some day other screens will be similar).

what this information is meant to convey

https://arctos.database.museum/tblbrowse.cfm?tbl=specimen_event

make sure that the labels fit.

That's one of the original use cases for field-level documentation, but we don't follow it (nor do we have the tools to make that mostly-realistic). Maybe that should be merged with the table browser in some way. Also, context - which varies from UI to UI - matters, unless we want to be very verbose everywhere (which brings on a whole new batch of usability problems).

In any case, I'd like to keep more of the very core documentation - "this table does THAT, this field means THING" - in the database itself, where it's available to APIs (including local UI). That's probably yet another Issue....

why "habitat" falls within this section of the stack and is not a "collecting event" attribute.

That's reflective of a lack of precision in recording events; events end up as "today, within a mile of here" and habitat within that event is "under a rotting log" and "bare vertical rock" and .... I doubt we can do much for legacy data, but we can certainly make recommendations to GPS-equipped collectors going forward.

"collecting method" and "collecting source",

Same as above, but also things like host being collected in a pitfall and parasites being collected in a Number Whatever Sieve (so not "fixable" via more precise place-time data).

Jegelewicz commented 2 years ago
data entry screen were structured in the same way the catalog record is

It won't be - one has a fixed structure (with some "extras"), the other is purely data driven and has no limitations.

I just mean the same order - right now locality is where collectors and identifiers are in the catalog record. If we could keep to the same general order, this would help, I think - but this is just one person's opinion.

dustymc commented 2 years ago

locality is where

... ever a user put it.

Screen Shot 2021-10-28 at 11 32 47 AM
Jegelewicz commented 2 years ago

Fair enough.

OK, here is the comparison for "collecting event" which is a misnomer, because it can also represent any of the other event types in https://arctos.database.museum/info/ctDocumentation.cfm?table=ctspecimen_event_type

Arctos Page/What it does Verbatim Event Location Verbatim Event Date Formatted Event Start Date Formatted Event End Date Event Remark Event Name Verbatim Coordinates Event Attribute Type Event Attribute Value Event Attribute Units Event Attribute Determiner Event Attribute Date Event Attribute Method Event Attribute Remark Locality Name Event ID Locality ID
Edit catalog record Verbatim Locality Verbatim Date Began Date/Time Ended Date/Time Collecting Event Remark Collecting Event Nickname As-entered Coordinates Event Attribute Type Event Attribute Value Event Attribute Units Event Attribute Determiner Event Attribute Date Event Attribute Method Event Attribute Remark locality_name N/A  
Catalog record Verbatim Locality Verbatim Date Event Date Event Date (to) Remark Event Nickname As Entered Attribute Value included in value if applicable Determiner Date Method Remark Locality Nickname N/A  
Data entry (old) Verbatim Locality VerbatimDate Begin End CollEvntRemk Event Nickname Coordinates (event and locality) N/A N/A N/A N/A N/A N/A N/A Locality Nickname Existing EventID  
Data entry (new) Verbatim Locality Verbatim Date Began Date Ended Date Collecting Event Remark Event Name all the coordinate stuff that is not decimal degrees N/A N/A N/A N/A N/A N/A N/A Locality Name Event ID Locality ID
Bulkloader builder verbatim_locality verbatim_date began_date ended_date coll_event_remarks collecting_event_name all the coordinate stuff that is not decimal degrees N/A N/A N/A N/A N/A N/A N/A locality_name collecting_event_id locality_id
Bulkload specimen-events verbatim_locality verbatim_date began_date ended_date coll_event_remarks collecting_event_name all the coordinate stuff that is not decimal degrees N/A N/A N/A N/A N/A N/A N/A locality_name N/A N/A  
Bulkload collecting events verbatim_locality verbatim_date began_date ended_date coll_event_remarks collecting_event_name N/A N/A N/A N/A N/A N/A N/A N/A locality_name N/A N/A 
Bulkload collecting event attributes N/A N/A N/A N/A N/A event_name N/A event_attribute_type event_attribute_value event_attribute_units event_determiner event_determined_date event_determination_method event_attribute_remark N/A N/A N/A

A few thoughts on this:

  1. Why are we including Event and Locality ID's as pick values when these could so easily change?
  2. Where in the heck does "associated species" fall in all of this? Is it part of the event stack or not?
  3. Why can't I enter event attributes as "extras"? It seems like we are discouraging expanding information related to the event because the only way to enter it is manually or through an event attribute bulkload (which assumes you have named events, which you cannot create in data entry). If this is technically too difficult, then OK, but if I can enter locality attributes as extras, it seems like event attributes should be no different.
dustymc commented 2 years ago

misnomer

No argument, but that's Major Surgery (and I'm not sure I have better names).

Why are we including Event and Locality ID's as pick values when these could so easily change?

@campmlc had some compelling reason, and the merge does handle the bulkloader (but not some spreadsheet on your desktop, of course).

Where in the heck does "associated species" fall in all of this?

https://arctos.database.museum/tblbrowse.cfm?tbl=coll_object_remark

Is it part of the event stack or not?

No - but it should be moved to specimen_event, I think. Just a migration miss IIRC.

I can enter locality attributes as extras

You cannot: https://arctos.database.museum/tblbrowse.cfm?tbl=bulkloader

Why can't I enter event attributes as "extras"?

Maybe plausible, certainly open for discussion, a few could be flattened as well, I think I'd prefer to pre-create named events and just use them (but IDK how that aligns with reality).

Jegelewicz commented 2 years ago
I can enter locality attributes as extras

You cannot: https://arctos.database.museum/tblbrowse.cfm?tbl=bulkloader

UMMMM

image

Jegelewicz commented 2 years ago

Also image

dustymc commented 2 years ago

"extras" - if you see it in the builder, it's not an extra.

Jegelewicz commented 2 years ago

Sorry for my poor use of "extras" - my argument is still the same, if I can do this for a locality, it seems that I should also be able to do it for an event, either directly or via "extras".

Jegelewicz commented 2 years ago

And finally - comparison for locality

Arctos Page/What it does Higher Geography Locality Name Locality ID Specific Locality Minimum Elevation Maximum Elevation Elevation Units Minimum Depth Maximum Depth Depth Units Locality Remark Coordinate Units Decimal Latitiude Decimal Longitude Maximum Coordinate Error Coordinate Error Units Datum Georeference Source Georeference Protocol WKT Media ID Locality Attribute Locality Attribute Value Locality Attribute Units Locality Attribute Determiner Locality Attribute Determined Date Locality Attribute Determined Method Locality Attribute Remark
Edit catalog record Higher Geography Locality Nickname   Specific Locality Min. Elev. Max. Elev. Elev. Unit Min. Depth Max. Depth Depth Units Locality Remarks N/A Decimal Latitude Decimal Longitude Max Error Max Error Units Datum Georeference Source Georeference Protocol WKTMediaID Type Value Units Determiner Date Method Remark
Catalog record Higher Geography Locality Nickname   Specific Locality Elevation (to) Elevation Elevation Depth (to) Depth Depth Locality Remarks N/A Coordinates Coordinates Error Error Datum Georeference Source Georeference Protocol   Attribute Value Units Determiner Date Method Remark
Data entry (old) Higher Geog Locality Nickname Existing LocalityID Spec Locality Elevation (min-max) between Elevation (min-max) between Elevation (min-max) between Depth (min-max) between Depth (min-max) between Depth (min-max) between LocalityRem Original lat/long Units Dec Lat Dec Long Max Error Max Error Datum Georeference Source Georeference Protocol WKTMediaID Type Value Unit Determiner Date Method Remark
Data entry (new) Higher Geography Locality Name Locality ID Specific Locality Minimum Elevation Maximum Elevation Elevation Units Minimum Depth Maximum Depth Depth Units Locality Remarks Coordinate Units Dec Lat Dec Long Max Error Max Error Units Datum Georeference Source Georeference Protocol WKTMediaID locality_attribute_type locality_attribute_value locality_attribute_unit locality_attribute_determiner locality_attribute_detr_date locality_attribute_detr_meth locality_attribute_remark
Bulkloader builder higher_geog locality_name locality_id spec_locality minimum_elevation maximum_elevation orig_elev_units min_depth max_depth depth_units locality_remarks orig_lat_long_units dec_lat dec_long max_error_distance max_error_units datum georeference_source georeference_protocol wkt_media_id locality_attribute_type locality_attribute_value locality_attribute_units locality_attribute_determiner locality_attribute_detr_date locality_attribute_detr_meth locality_attribute_remark
Bulkload specimen-events   locality_name   spec_locality minimum_elevation maximum_elevation orig_elev_units min_depth max_depth depth_units locality_remarks orig_lat_long_units dec_lat dec_long max_error_distance max_error_units datum georeference_source georeference_protocol wkt_media_id N/A N/A N/A N/A N/A N/A N/A
Bulkload collecting events higher_geog     spec_locality minimum_elevation maximum_elevation orig_elev_units min_depth max_depth depth_units locality_remarks   dec_lat dec_long max_error_distance max_error_units datum georeference_source georeference_protocol N/A N/A N/A N/A N/A N/A N/A N/A
Bulkload locality                                                      
Bulkload locality attributes N/A locality_name N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A attribute_type attribute_value attribute_units attribute_determiner determined_date determination_method attribute_remark

I need to add in the names for bulkload locality - but the tool 404s for me right now. See #4053

A few things here.

  1. No higher geography field in the specimen event loader
  2. Datum is not included on the catalog record
  3. I think we really should be recording who georeferenced and when they did it
  4. When the georeferencing happens "behind the scenes" (coordinates other than decimal degrees are entered and used to create decimal degree values), the who would be "Arctos"?
dustymc commented 2 years ago

No higher geography field in the specimen event loader

Screen Shot 2021-10-28 at 4 30 38 PM

Datum is not included on the catalog record

Screen Shot 2021-10-28 at 4 31 54 PM

who georeferenced and when t

File a CT request

"behind the scenes" (coordinates other than decimal degrees are entered and used to create decimal degree values), the who would be "Arctos"?

There's only one possibility, we gain nothing by recording it.

Jegelewicz commented 2 years ago

No higher geography field in the specimen event loader

Get a template - there is no higher geography in it.

Jegelewicz commented 2 years ago

Datum is not included on the catalog record

Screen Shot 2021-10-28 at 4 31 54 PM

HMMMM I swear I saw one with coordinates but no datum - is it possible that it gets hidden when it is "unknown"? Guess not, I'm blaming that on late lunch...

Jegelewicz commented 2 years ago

A revised summary

Locality

Arctos Page/What it does Higher Geography Locality Name Locality ID Specific Locality Minimum Elevation Maximum Elevation Elevation Units Minimum Depth Maximum Depth Depth Units Locality Remark Coordinate Units Decimal Latitiude Decimal Longitude Maximum Coordinate Error Coordinate Error Units Datum Georeference Source Georeference Protocol WKT Media ID Locality Attribute Locality Attribute Value Locality Attribute Units Locality Attribute Determiner Locality Attribute Determined Date Locality Attribute Determined Method Locality Attribute Remark
Edit catalog record Higher Geography Locality Nickname N/A Specific Locality Min. Elev. Max. Elev. Elev. Unit Min. Depth Max. Depth Depth Units Locality Remarks N/A Decimal Latitude Decimal Longitude Max Error Max Error Units Datum Georeference Source Georeference Protocol WKTMediaID Type Value Units Determiner Date Method Remark
Catalog record Higher Geography Locality Nickname N/A Specific Locality Elevation (to) Elevation Elevation Depth (to) Depth Depth Locality Remarks N/A Coordinates Coordinates Error Error   Georeference Source Georeference Protocol   Attribute Value Units Determiner Date Method Remark
Data entry (old) Higher Geog Locality Nickname Existing LocalityID Spec Locality Elevation (min-max) between Elevation (min-max) between Elevation (min-max) between Depth (min-max) between Depth (min-max) between Depth (min-max) between LocalityRem Original lat/long Units Dec Lat Dec Long Max Error Max Error Datum Georeference Source Georeference Protocol WKTMediaID Type Value Unit Determiner Date Method Remark
Data entry (new) Higher Geography Locality Name Locality ID Specific Locality Minimum Elevation Maximum Elevation Elevation Units Minimum Depth Maximum Depth Depth Units Locality Remarks Coordinate Units Dec Lat Dec Long Max Error Max Error Units Datum Georeference Source Georeference Protocol WKTMediaID locality_attribute_type locality_attribute_value locality_attribute_unit locality_attribute_determiner locality_attribute_detr_date locality_attribute_detr_meth locality_attribute_remark
Bulkloader builder higher_geog locality_name locality_id spec_locality minimum_elevation maximum_elevation orig_elev_units min_depth max_depth depth_units locality_remarks orig_lat_long_units dec_lat dec_long max_error_distance max_error_units datum georeference_source georeference_protocol wkt_media_id locality_attribute_type locality_attribute_value locality_attribute_units locality_attribute_determiner locality_attribute_detr_date locality_attribute_detr_meth locality_attribute_remark
Bulkload specimen-events higher_geog locality_name N/A spec_locality minimum_elevation maximum_elevation orig_elev_units min_depth max_depth depth_units locality_remarks orig_lat_long_units dec_lat dec_long max_error_distance max_error_units datum georeference_source georeference_protocol wkt_media_id N/A N/A N/A N/A N/A N/A N/A
Bulkload collecting events higher_geog N/A N/A spec_locality minimum_elevation maximum_elevation orig_elev_units min_depth max_depth depth_units locality_remarks N/A dec_lat dec_long max_error_distance max_error_units datum georeference_source georeference_protocol N/A N/A N/A N/A N/A N/A N/A N/A
Bulkload locality higher_geog locality_name N/A spec_locality minimum_elevation maximum_elevation orig_elev_units min_depth max_depth depth_units locality_remarks N/A dec_lat dec_long max_error_distance max_error_units datum georeference_source georeference_protocol wkt_media_id N/A N/A N/A N/A N/A N/A N/A
Bulkload locality attributes N/A locality_name N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A attribute_type attribute_value attribute_units attribute_determiner determined_date determination_method attribute_remark
Search Continent/Ocean, Sea, Country, State/Province, USGS Quad Map, County, Island Group, Island, Geographic Feature, Drainage N/A N/A Specific Locality Elevation Elevation Elevation N/A N/A N/A Locality Remarks N/A N/A N/A Coordinate Precision Coordinate Precision N/A Georeference Source N/A N/A Locality Attribute 1 Locality Attribute 1 Value Locality Attribute 1 Unit Locality Attribute 1 Determiner N/A Locality Attribute 1 Method Locality Attribute 1 Remark

Event

Arctos Page/What it does Verbatim Event Location Verbatim Event Date Formatted Event Start Date Formatted Event End Date Event Remark Event Name Verbatim Coordinates Event Attribute Type Event Attribute Value Event Attribute Units Event Attribute Determiner Event Attribute Date Event Attribute Method Event Attribute Remark Locality Name Event ID Locality ID
Edit catalog record Verbatim Locality Verbatim Date Began Date/Time Ended Date/Time Collecting Event Remark Collecting Event Nickname As-entered Coordinates Event Attribute Type Event Attribute Value Event Attribute Units Event Attribute Determiner Event Attribute Date Event Attribute Method Event Attribute Remark locality_name N/A N/A
Catalog record Verbatim Locality Verbatim Date Event Date Event Date (to) Remark Event Nickname As Entered Attribute Value included in value if applicable Determiner Date Method Remark Locality Nickname N/A N/A
Data entry (old) Verbatim Locality VerbatimDate Begin End CollEvntRemk Event Nickname Coordinates (event and locality) N/A N/A N/A N/A N/A N/A N/A Locality Nickname Existing EventID N/A
Data entry (new) Verbatim Locality Verbatim Date Began Date Ended Date Collecting Event Remark Event Name all the coordinate stuff that is not decimal degrees N/A N/A N/A N/A N/A N/A N/A Locality Name Event ID Locality ID
Bulkloader builder verbatim_locality verbatim_date began_date ended_date coll_event_remarks collecting_event_name all the coordinate stuff that is not decimal degrees N/A N/A N/A N/A N/A N/A N/A locality_name collecting_event_id locality_id
Bulkload specimen-events verbatim_locality verbatim_date began_date ended_date coll_event_remarks collecting_event_name all the coordinate stuff that is not decimal degrees N/A N/A N/A N/A N/A N/A N/A locality_name N/A N/A
Bulkload collecting events verbatim_locality verbatim_date began_date ended_date coll_event_remarks collecting_event_name N/A N/A N/A N/A N/A N/A N/A N/A locality_name N/A N/A
Bulkload collecting event attributes N/A N/A N/A N/A N/A event_name N/A event_attribute_type event_attribute_value event_attribute_units event_determiner event_determined_date event_determination_method event_attribute_remark N/A N/A N/A
Search Verbatim Locality Verbatim Date Collected On or After Collected On or Before Collecting Event Remark Search N/A Event Attributes N/A N/A N/A N/A N/A N/A N/A N/A N/A

Event Assignment

Arctos Page/What it does Assigned Event Type Event Assigned By Event Assigned Date Event Verification Event Verified By Event Verification Date Event Assignment Remark Event Habitat Event Source Event Method
Edit catalog record Specimen/Event Type Event Determiner Determined Date Verification Status Verified By Verified Date Specimen/Event Remark Habitat Collecting Source Collecting Method
Catalog record Event Type Event Type: assigned by Event Date Event Verification: Event Verification: verified by Event Verification: on Event Details: remark Event Details: habitat Event Details: Collecting Source Event Details: collecting method
Data entry (old) Specimen/Event Type Event Determiner Detr. Date VerificationStatus N/A N/A Specimen/Event Remark Habitat Coll. Src. Coll. Meth.
Data entry (new) Event Type Event Determiner Det. Date VerificationStatus N/A N/A Event Remark Habitat Collecting Source Collecting Method
Bulkloader builder specimen_event_type event_assigned_by_agent event_assigned_date verificationstatus N/A N/A specimen_event_remark habitat collecting_source collecting_method
Bulkload specimen-events specimen_event_type assigned_by_agent assigned_date verificationstatus N/A N/A specimen_event_remark habitat collecting_source collecting_method
Search Record/Event Type N/A N/A Verification Status N/A N/A Record/Event Remark Habitat * in Locality Collecting Source Collecting Method