ArctosDB / arctos

Arctos is a museum collections management system
https://arctos.database.museum
60 stars 13 forks source link

Code Table Request - DMNS: Denver Museum of Nature and Science #5911

Closed dustymc closed 1 month ago

dustymc commented 1 year ago

Instructions

This is a template to facilitate communication with the Arctos Code Table Committee. Submit a separate request for each relevant value. This form is appropriate for exploring how data may best be stored, for adding vocabulary, or for updating existing definitions.

Reviewing documentation before proceeding will result in a more enjoyable experience.


Initial Request

Goal: Describe what you're trying to accomplish. This is the only necessary step to start this process. The Committee is available to assist with all other steps. Please clearly indicate any uncertainty or desired guidance if you proceed beyond this step.

All DMNS: Denver Museum of Nature and Science should be replaced with other ID type = other identifier and issued by agent Denver Museum of Nature & Science

Proposed Value: Proposed new value. This should be clear and compatible with similar values in the relevant table and across Arctos.

Proposed Definition: Clear, complete, non-collection-type-specific functional definition of the value. Avoid discipline-specific terminology if possible, include parenthetically if unavoidable.

Context: Describe why this new value is necessary and existing values are not.

Table: Code Tables are http://arctos.database.museum/info/ctDocumentation.cfm. Link to the specific table or value. This may involve multiple tables and will control datatype for Attributes. OtherID requests require BaseURL (and example) or explanation. Please ask for assistance if unsure.

Collection type: Some code tables contain collection-type-specific values. collection_cde may be found from https://arctos.database.museum/home.cfm

Priority: Please describe the urgency and/or choose a priority-label to the right. You should expect a response within two working days, and may utilize Arctos Contacts if you feel response is lacking.

Available for Public View: Most data are by default publicly available. Describe any necessary access restrictions.

Project: Add the issue to the Code Table Management Project.

Discussion: Please reach out to anyone who might be affected by this change. Leave a comment or add this to the Committee agenda if you believe more focused conversation is necessary.

Approval

All of the following must be checked before this may proceed.

The How-To Document should be followed. Pay particular attention to terminology (with emphasis on consistency) and documentation (with emphasis on functionality).

Rejection

If you believe this request should not proceed, explain why here. Suggest any changes that would make the change acceptable, alternate (usually existing) paths to the same goals, etc.

  1. Can a suitable solution be found here? If not, proceed to (2)
  2. Can a suitable solution be found by Code Table Committee discussion? If not, proceed to (3)
  3. Take the discussion to a monthly Arctos Working Group meeting for final resolution.

Implementation

Once all of the Approval Checklist is appropriately checked and there are no Rejection comments, or in special circumstances by decree of the Arctos Working Group, the change may be made.

Review everything one last time. Ensure the How-To has been followed. Ensure all checks have been made by appropriate personnel.

Make changes as described above. Ensure the URL of this Issue is included in the definition.

Close this Issue.

DO NOT modify Arctos Authorities in any way before all points in this Issue have been fully addressed; data loss may result.

Special Exemptions

In very specific cases and by prior approval of The Committee, the approval process may be skipped, and implementation requirements may be slightly altered. Please note here if you are proceeding under one of these use cases.

  1. Adding an existing term to additional collection types may proceed immediately and without discussion, but doing so may also subject users to future cleanup efforts. If time allows, please review the term and definition as part of this step.
  2. The Committee may grant special access on particular tables to particular users. This should be exercised with great caution only after several smooth test cases, and generally limited to "taxonomy-like" data such as International Commission on Stratigraphy terminology.
dustymc commented 1 year ago

I will plan on proceeding with this about 2023-03-27 if there are no objections.

I will proceed immediately upon approval of each of the involved collections.

Data: temp_dmns_denver_museum_of_nature_and_science.csv.zip

Summary: guid_prefix numrecs
CHAS:Bird 301
DMNS:Bird 42
DMNS:Mamm 5
MSB:Mamm 209
MVZ:Mamm 1
UCM:Herp 543

Users: @ebraker @mkoo @campmlc @cjconroy @wellerjes @AdrienneRaniszewski @acdoll @jldunnum @jrdemboski @droberts49

See also https://github.com/ArctosDB/arctos/issues/5771

cjconroy commented 1 year ago

OK for MVZ Mammals

campmlc commented 1 year ago

MSB:Mamm records need to be changed to link to the DMNS:Mamm equivalent, with this numeric value completing the guid. Also relationships need to be created for "same individual" as between these collections. @jldunnum

cjconroy commented 1 year ago

I asked in another thread whether we needed to edit relationships and the response was No. So, do we really need to track down every one of these and add "same individual as"? Can't that be magicked?

Jegelewicz commented 1 year ago

edit relationships and the response was No. So, do we really need to track down every one of these and add "same individual as"? Can't that be magicked?

If a collection is confident that the relationships should be made, I feel pretty sure @dustymc can magic them - but there must be some reason these were entered the way they were and I think that a little investigation is warranted.

cjconroy commented 1 year ago

This sounds so illogical. Pretty much all the other ID's for us indicate a part is here and a part is there. Not clear what that has to do with confidence. Do you mean actually checking with the other collection to confirm they still have the part? If that is part of the process, shouldn't that be mentioned in all of these emails we are getting as an important step?

dustymc commented 1 year ago

Yes I can magic or semi-magic (eg fill the ID loader and unloader), but I can't usefully evaluate. Confidently-asserted bad date is horrid to deal with; if there aren't resources to evaluate then I'd recommend going with the initial proposal, which is crafted to not dig any holes any deeper (nor to make them any shallower, unfortunately).

I would always assume a "11096" with a type like this DOES NOT correspond with https://arctos.database.museum/guid/DMNS:Mamm:11096 (or someone would have used that) - things get dumped into the 'one of the dozen or hundreds of things issued by giant institution over the decades' baskets because that's all that's known.

campmlc commented 1 year ago

The MSB records are all from a single grad student project who was working with both institutions, and she did the data entry. It is likely she did not understand that using DMNS:Mamm was the preferred option. Tissues are at MSB and vouchers are at DMNS. All 209 records that need to be converted to the DMNS:Mamm identifier with relationship same individual as.

campmlc commented 1 year ago

It also looks like in some cases, the mammal voucher is at MSB, and the tissues are at DMNS, or tissues were shared between institutions. In either case, in addition to the DMNS identifier, there is a DZTM: Denver Zoology Tissue Mammal identifier referencing the Denver tissue collection number. This is an explicit ID that needs to be distinct from just other ID issued by DMNS. Please hold off of doing anything with the MSB records until this can be reviewed, discussed and updated by MSB personnel. @jldunnum @AdrienneRaniszewski

campmlc commented 1 year ago

6004 #6002

acdoll commented 1 year ago

but there must be some reason these were entered the way they were

Because relationships weren't a thing when they were entered and the options of otherID_type were substantially limited back then.

she did not understand that using DMNS:Mamm was the preferred option

again, in her defense, that wasn't always an option

For lack of a better place to make this comment, these changes are going to fully screw up our label printing in the reporter. Of course, we have to recreate all of our labels/tags/reports in the new reporter, so I guess we'll figure that out then...

dustymc commented 1 year ago

label printing

File an Issue with specifics - should be straightforward, hopefully I can just magic it all.

(And FWIW Arctos was born with inter-Arctos relationships - hopefully we've gotten a lot better at documenting and communicating.)

ebraker commented 1 year ago

OK for UCM

Jegelewicz commented 1 year ago

All of these should probably be reviewed to see if they should be matched to a DMNS:Herp, DMNS:Mamm, or DMNS:Bird catalog record instead of this generic issuer.

ebraker commented 1 year ago

@Jegelewicz I reviewed UCM's when I migrated to Arctos. So it would just be updates if anything has changed for us at least.

Jegelewicz commented 1 year ago

@ebraker and @acdoll any reason they wouldn't match a DMNS:Herp record?

ebraker commented 1 year ago

We received the entirety of the DMNS herp collection (or close to it) several decades ago. My understanding is that current DMNS:Herp is a new collection of incidental/bycatch material. So there wouldn't be crossover.

wellerjes commented 10 months ago

OK for CHAS

dustymc commented 10 months ago

OK for CHAS

Thanks, done!

OK for UCM

Bah, sorry I somehow missed that, done!

Fresh (pre-update) data:

temp_dmns_denver_museum_of_nature_and_science.csv.zip

Summary:

 guid_prefix | numrecs 
-------------+---------
 CHAS:Bird   |     285
 DMNS:Bird   |      42
 DMNS:Mamm   |       5
 MSB:Mamm    |     209
 UCM:Herp    |     543
acdoll commented 4 months ago

All of these should probably be reviewed to see if they should be matched to a DMNS:Herp, DMNS:Mamm, or DMNS:Bird catalog record instead of this generic issuer.

Yes, most of these should use their specific collection identifier.

My understanding is that current DMNS:Herp is a new collection of incidental/bycatch material. So there wouldn't be crossover.

Correct. - these are ones that should probably get the generic identifier type. The same goes for all of the other non-integer identifiers on this list.

dustymc commented 1 month ago

This is actively causing confusion, from today's' error logs:

INSERT INTO coll_obj_other_id_num( collection_object_id, other_id_type, display_value, id_references, issued_by_agent_id, remarks ) VALUES ( 22933293, 'DMNS: Denver Museum of Nature and Science', 'https://arctos.database.museum/guid/DMNS:Inv:388', 'same lot as', null, null )

Summary of remaining data:


 guid_prefix | count 
-------------+-------
 DMNS:Bird   |    41
 DMNS:Mamm   |     5
 MSB:Mamm    |   209
(3 rows)

The proposed update retains exactly the information available in the type, there is no functional reason to delay this, and there is no reason to clean or correct anything before proceeding; that can be accomplished at any time (including never, in which case the data will continue to carry precisely as much information as they do now).

Also of note: The agent carries ~3x the number of identifiers the type does, there is significant denormalization caused by this type, it is preventing users from finding data.

@mkoo help?

campmlc commented 1 month ago

I believe all of the MSB links should have an updated link to the correct Arctos GUID for DMNS:Mamm. If there is an easy way to check that, then the MSB identifiers can be converted or removed as redundant.

dustymc commented 1 month ago

@campmlc if you'll tell me what data you need I can get it for you, but I still can't see any reason that cleanup needs to block removing this denormalizer.

campmlc commented 1 month ago

Cleanup doesn't need to block this, but it would be helpful if you could send me a download of all MSB records with DMNS identifiers and also DMNS:Mamm Arctos guid identifiers - hopefully they are the same

dustymc commented 1 month ago

@campmlc here you go

temp_dmns_denver_museum_of_nature_and_science_plus_arg.csv.zip

@acdoll OK to upgrade DMNS?

acdoll commented 1 month ago

Yes, please do