Catalog Item Type observation should be removed from the code table

[Jegelewicz]

From today's Observation Interest Group Meeting

observations should all be converted to either human observation or machine observation

Originally posted by @Jegelewicz in https://github.com/ArctosDB/arctos/issues/2432#issuecomment-1376247660

Affected code table - cataloged item type

Affected term - observation

Steps to make this happen

• [x] Get a master list of all catalog records using observation as catalog item type
• [ ] change these to either human observation or machine observation
• [ ] remove observation from the code table

[Jegelewicz]

Collections using catalog item type = observation:

MSBObs:Mamm MVZObs:Herp MVZObs:Mamm UAMObs:Ento UAMObs:Mamm

@jldunnum @atrox @cjconroy @DerekSikes @amgunderson

Review at https://docs.google.com/spreadsheets/d/13AEgtP8YHM7gW6_VQWyN25oE010vzpMy-x5Ote50GfM/edit?usp=sharing

Observation interest group suggests that all records using catalog record type = observation and

1. with associated media and no parts or media parts be changed to "machine observation"
2. with no associated media OR parts be changed to "human observation"
3. with any other combination of parts/media requires review.

@ccicero anything to add?

[jldunnum]

I'm fine with the change. We actually haven't really used this for anything yet but are hoping to start adding bat call data here in the not too distant future.

---

Jonathan L. Dunnum Ph.D. (he, him, his) Senior Collection Manager Division of Mammals, Museum of Southwestern Biology University of New Mexico Albuquerque, NM 87131 (505) 277-9262 Fax (505) 277-1351

Chair, Systematic Collections Committee, American Society of Mammalogists Latin American Fellowship Committee, ASM

MSB Mammals website: http://www.msb.unm.edu/mammals/index.html Facebook: http://www.facebook.com/MSBDivisionofMammals

Shipping Address: Museum of Southwestern Biology Division of Mammals University of New Mexico CERIA Bldg 83, Room 204 Albuquerque, NM 87131

---

From: Teresa Mayfield-Meyer @.> Sent: Monday, February 13, 2023 2:55 PM To: ArctosDB/arctos @.> Cc: Jonathan Dunnum @.>; Mention @.> Subject: Re: [ArctosDB/arctos] Catalog Item Type observation should be removed from the code table (Issue #5459)

[EXTERNAL]

Collections using catalog item type = observation:

MSBObs:Mamm MVZObs:Herp MVZObs:Mamm UAMObs:Ento UAMObs:Mamm

@jldunnumhttps://github.com/jldunnum @Atroxhttps://github.com/Atrox @cjconroyhttps://github.com/cjconroy @DerekSikeshttps://github.com/DerekSikes @amgundersonhttps://github.com/amgunderson

Review at https://docs.google.com/spreadsheets/d/13AEgtP8YHM7gW6_VQWyN25oE010vzpMy-x5Ote50GfM/edit?usp=sharing

Observation interest group suggests that all records using catalog record type = observation and

1. with associated media and no parts or media parts be changed to "machine observation"
2. with no associated media OR parts be changed to "human observation"
3. with any other combination of parts/media requires review.

@ccicerohttps://github.com/ccicero anything to add?

— Reply to this email directly, view it on GitHubhttps://github.com/ArctosDB/arctos/issues/5459#issuecomment-1428742434, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AED2PA456V3SI6P2TYNB3PTWXKUV5ANCNFSM6AAAAAATV3ERPM. You are receiving this because you were mentioned.Message ID: @.***>

[dustymc]

haven't used this

https://arctos.database.museum/search.cfm?guid_prefix=MSBObs%3AMamm&customoidoper=BETWEEN&cataloged_item_type=observation

[jldunnum]

Well yeah, basically put three records in to try it out when we were going through the discussions of digitizing the various events associated with the Mexican wolf recovery program.

---

Jonathan L. Dunnum Ph.D. (he, him, his) Senior Collection Manager Division of Mammals, Museum of Southwestern Biology University of New Mexico Albuquerque, NM 87131 (505) 277-9262 Fax (505) 277-1351

Chair, Systematic Collections Committee, American Society of Mammalogists Latin American Fellowship Committee, ASM

MSB Mammals website: http://www.msb.unm.edu/mammals/index.html Facebook: http://www.facebook.com/MSBDivisionofMammals

Shipping Address: Museum of Southwestern Biology Division of Mammals University of New Mexico CERIA Bldg 83, Room 204 Albuquerque, NM 87131

---

From: dustymc @.> Sent: Monday, February 13, 2023 3:17 PM To: ArctosDB/arctos @.> Cc: Jonathan Dunnum @.>; Mention @.> Subject: Re: [ArctosDB/arctos] Catalog Item Type observation should be removed from the code table (Issue #5459)

[EXTERNAL]

haven't used this

https://arctos.database.museum/search.cfm?guid_prefix=MSBObs%3AMamm&customoidoper=BETWEEN&cataloged_item_type=observation

— Reply to this email directly, view it on GitHubhttps://github.com/ArctosDB/arctos/issues/5459#issuecomment-1428770082, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AED2PA6JHKPFCX4N2AXP3GDWXKXIVANCNFSM6AAAAAATV3ERPM. You are receiving this because you were mentioned.Message ID: @.***>

[amgunderson]

Are "human observation" and "machine observation" GBIF's terms? And we are doing this to accommodate GBIF? I don't object but those terms are awkward and most of those with media are also human observations.

[Jegelewicz]

Are "human observation" and "machine observation" GBIF's terms?

Actually they are Darwin Core, but yes, GBIF uses them.

And we are doing this to accommodate GBIF?

Yes - the record type "observation" is not recognized by GBIF and any dataset with a row using it in BasisOfRecord, will not be published.

most of those with media are also human observations.

True, but because they offer something that one can independently review (the image or sound file) they have another level of potential use.

[ccicero]

Those criteria look good to me for converting 'observation' to either 'machine observation' or 'human observation'

[Jegelewicz]

@DerekSikes are you OK with this for UAMObs:Ento?

[campmlc]

@jldunnum OK to change those MSB:MammObs wolf records to human or machine observation? I'm assuming those need to be added as collecting events to MSB:Mamm:61157, unless we have blood samples somewhere? What are the media parts?

[DerekSikes]

Reading the Darwin core definitions: https://dwc.tdwg.org/terms/#humanobservation

makes me wonder how we will parse our UAMObs:Ento 40,000 records - could someone explain how that will happen?

Is it records with media = machine? Records without media = human?

How will that be enforced? What if I forget and make an observation that has media and call it human. Or what if I add media to a human observation that didn't have media before?

On Thu, Feb 16, 2023 at 1:32 PM Mariel Campbell @.***> wrote:

@jldunnum https://github.com/jldunnum OK to change those MSB:MammObs wolf records to human or machine observation? I'm assuming those need to be added as collecting events to MSB:Mamm:61157, unless we have blood samples somewhere? What are the media parts?

— Reply to this email directly, view it on GitHub https://github.com/ArctosDB/arctos/issues/5459#issuecomment-1433807602, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACFNUMYHU7BXZ37BTK7IPPLWX2TIDANCNFSM6AAAAAATV3ERPM . You are receiving this because you were mentioned.Message ID: @.***>

--

+++++++++++++++++++++++++++++++++++ Derek S. Sikes, Curator of Insects, Professor of Entomology University of Alaska Museum (UAM), University of Alaska Fairbanks 1962 Yukon Drive, Fairbanks, AK 99775-6960 @.*** phone: 907-474-6278 he/him/his University of Alaska Museum https://www.uaf.edu/museum/collections/ento/

• search 357,704 digitized arthropod records http://arctos.database.museum/uam_ento +++++++++++++++++++++++++++++++++++

Interested in Alaskan Entomology? Join the Alaska Entomological Society and / or sign up for the email listserv "Alaska Entomological Network" at http://www.akentsoc.org/contact_us

[Jegelewicz]

Is it records with media = machine? Records without media = human?

yes

How will that be enforced?

It won't. This proposal is only to change things currently using "observation" to one or the other.

What if I forget and make an observation that has media and call it human.

Then you will have selected poorly, but in the grand scheme of things it isn't that big of a deal at least you will have selected a term that GBIF will accept and your observation data can be published there.

Or what if I add media to a human observation that didn't have media before?

You can change the collection object type to reflect the new information.

Or not. If you aren't concerned about publishing to GBIF then this may not matter to you at all and selecting one type of observation or another won't bother you. If you are publishing to GBIF, then it does matter because people parse data there based upon whether they can independently review evidence or not using these two terms.

[DerekSikes]

Teresa,

Thanks! It appears our observation data are already in GBIF eg https://www.gbif.org/occurrence/896907839

with original basisofrecord = HumanObservation

???

-D

On Thu, Feb 16, 2023 at 1:51 PM Teresa Mayfield-Meyer < @.***> wrote:

Is it records with media = machine? Records without media = human?

yes

How will that be enforced?

It won't. This proposal is only to change things currently using "observation" to one or the other.

What if I forget and make an observation that has media and call it human.

Then you will have selected poorly, but in the grand scheme of things it isn't that big of a deal at least you will have selected a term that GBIF will accept and your observation data can be published there.

Or what if I add media to a human observation that didn't have media before?

You can change the collection object type to reflect the new information.

[image: image] https://user-images.githubusercontent.com/5725767/219504369-7809bd67-8fe7-4924-b4ac-7f15263dec5e.png

Or not. If you aren't concerned about publishing to GBIF then this may not matter to you at all and selecting one type of observation or another won't bother you. If you are publishing to GBIF, then it does matter because people parse data there based upon whether they can independently review evidence or not using these two terms.

— Reply to this email directly, view it on GitHub https://github.com/ArctosDB/arctos/issues/5459#issuecomment-1433844243, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACFNUM3DKNGJUV66WPZ7XYDWX2VNZANCNFSM6AAAAAATV3ERPM . You are receiving this because you were mentioned.Message ID: @.***>

--

+++++++++++++++++++++++++++++++++++ Derek S. Sikes, Curator of Insects, Professor of Entomology University of Alaska Museum (UAM), University of Alaska Fairbanks 1962 Yukon Drive, Fairbanks, AK 99775-6960 @.*** phone: 907-474-6278 he/him/his University of Alaska Museum https://www.uaf.edu/museum/collections/ento/

• search 357,704 digitized arthropod records http://arctos.database.museum/uam_ento +++++++++++++++++++++++++++++++++++

Interested in Alaskan Entomology? Join the Alaska Entomological Society and / or sign up for the email listserv "Alaska Entomological Network" at http://www.akentsoc.org/contact_us

[DerekSikes]

But I have no objections to this change.

-D

On Thu, Feb 16, 2023 at 1:56 PM Derek Sikes @.***> wrote:

Teresa,

Thanks! It appears our observation data are already in GBIF eg https://www.gbif.org/occurrence/896907839

with original basisofrecord = HumanObservation

???

-D

On Thu, Feb 16, 2023 at 1:51 PM Teresa Mayfield-Meyer < @.***> wrote:

Is it records with media = machine? Records without media = human?

yes

How will that be enforced?

It won't. This proposal is only to change things currently using "observation" to one or the other.

What if I forget and make an observation that has media and call it human.

Then you will have selected poorly, but in the grand scheme of things it isn't that big of a deal at least you will have selected a term that GBIF will accept and your observation data can be published there.

Or what if I add media to a human observation that didn't have media before?

You can change the collection object type to reflect the new information.

[image: image] https://user-images.githubusercontent.com/5725767/219504369-7809bd67-8fe7-4924-b4ac-7f15263dec5e.png

Or not. If you aren't concerned about publishing to GBIF then this may not matter to you at all and selecting one type of observation or another won't bother you. If you are publishing to GBIF, then it does matter because people parse data there based upon whether they can independently review evidence or not using these two terms.

— Reply to this email directly, view it on GitHub https://github.com/ArctosDB/arctos/issues/5459#issuecomment-1433844243, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACFNUM3DKNGJUV66WPZ7XYDWX2VNZANCNFSM6AAAAAATV3ERPM . You are receiving this because you were mentioned.Message ID: @.***>

--

+++++++++++++++++++++++++++++++++++ Derek S. Sikes, Curator of Insects, Professor of Entomology University of Alaska Museum (UAM), University of Alaska Fairbanks 1962 Yukon Drive, Fairbanks, AK 99775-6960 @.*** phone: 907-474-6278 he/him/his University of Alaska Museum https://www.uaf.edu/museum/collections/ento/

• search 357,704 digitized arthropod records http://arctos.database.museum/uam_ento +++++++++++++++++++++++++++++++++++

Interested in Alaskan Entomology? Join the Alaska Entomological Society and / or sign up for the email listserv "Alaska Entomological Network" at http://www.akentsoc.org/contact_us

--

+++++++++++++++++++++++++++++++++++ Derek S. Sikes, Curator of Insects, Professor of Entomology University of Alaska Museum (UAM), University of Alaska Fairbanks 1962 Yukon Drive, Fairbanks, AK 99775-6960 @.*** phone: 907-474-6278 he/him/his University of Alaska Museum https://www.uaf.edu/museum/collections/ento/

• search 357,704 digitized arthropod records http://arctos.database.museum/uam_ento +++++++++++++++++++++++++++++++++++

Interested in Alaskan Entomology? Join the Alaska Entomological Society and / or sign up for the email listserv "Alaska Entomological Network" at http://www.akentsoc.org/contact_us

[Jegelewicz]

@DerekSikes I think that is because Dusty has been changing anything that is "observation" to "human observation" in the process of building the Darwin Core data for the IPT.

Thanks

[Jegelewicz]

@dustymc we have buy-in from the collections using "observation". Can we move forward doing this and then see what is left to review?

All records currently using catalog record type = observation and

1. with associated media and no parts except media parts be changed to "machine observation"
2. with no associated media OR parts be changed to "human observation"

[ccicero]

@dustymc can you also see if there are any other non-observational types (e.g., 'specimen') in Obs collections that either have no part or part of media - MVZObs:Bird, etc.?

[Jegelewicz]

@dustymc any chance we can get this done? https://github.com/ArctosDB/arctos/issues/5459#issuecomment-1433870048

[dustymc]

I'm not sure how to move on this - want to generate CSV and get the collections to sign off or something? Not sure the recipe above makes much sense to me, but they ain't my data so it doesn't have to.

Second, I'm not sure how to sort/prioritize non-dev issues, this was not on my radar at all, clever ideas appreciated.

[Jegelewicz]

Code table says let's just do this!

[Jegelewicz]

want to generate CSV and get the collections to sign off or something?

We have done that. See responses above. Let's do this before we have more users to poll.

[campmlc]

I support moving ahead with this. Can I check a box?

[dustymc]

Sorry, I got completely lost in the above. Here's fresh data.

temp_obs_citype.csv.zip

I did this:

update temp_obs_citype set updateto='preserved specimen' where parts not in ('NULL','media','media |media');
update temp_obs_citype set updateto='human observation' where media ='no' and parts in ('NULL','media','media |media');
update temp_obs_citype set updateto='machine observation' where media ='yes' and updateto is null;

which leaves this:

           parts            | media |      updateto       
----------------------------+-------+---------------------
 DNA                        | no    | preserved specimen
 ear clip                   | no    | preserved specimen
 heart, liver |heart, liver | no    | preserved specimen
 media                      | no    | human observation
 media                      | yes   | machine observation
 media |media               | no    | human observation
 media |media               | yes   | machine observation
 NULL                       | no    | human observation
 NULL                       | yes   | machine observation
 pellet                     | no    | preserved specimen
 tail                       | yes   | preserved specimen
 tissue                     | no    | preserved specimen
 tissue                     | yes   | preserved specimen
 whole organism             | no    | preserved specimen
 whole organism             | yes   | preserved specimen
(15 rows)

please let me know where I got lost or when I can make the updates.

[campmlc]

For MSB, the usage of media as part to replace observation is complicated, difficult, and doesn't accurately reflect what we have or do not have in collection. This requires a separate discussion. But for the purposes of this issue to standardize specimen type, the MSB records can move forward with the following corrections:

MSB:Host:1240 human observation
MSBObs:Herp:2 human observation MSBObs:Herp:1 human observation

@jtgiermakowski please review correction to the part and event types in the MSBObs:Herp records.

[Jegelewicz]

I changed MSB:Host:1240 to human observation

[Jegelewicz]

The pellet one should get changed to MaterialEntity when that is available.

[campmlc]

Can you explain what the difference is between preserved specimen and MaterialEntity? If we have tissue only, should it be the latter?

On Mon, Jul 24, 2023 at 1:03 PM Teresa Mayfield-Meyer < @.***> wrote:

• [EXTERNAL]*

The pellet one should get changed to MaterialEntity when that is available.

— Reply to this email directly, view it on GitHub https://github.com/ArctosDB/arctos/issues/5459#issuecomment-1648458367, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADQ7JBDKXKIK6BBKGZ5CZCLXR3BJXANCNFSM6AAAAAATV3ERPM . You are receiving this because you commented.Message ID: @.***>

[Jegelewicz]

If we have tissue only, should it be the latter?

That depends upon your definition of "specimen". I cannot tell you how loudly and often I have complained about the Darwin Core definition of "PreservedSpecimen" but people are attached to it and because there is no agreed-upon definition of "specimen" it remains vague. I don't know that it will make a huge difference over the long term, but making a choice and being consistent seems like a good idea. Because GBIF will not accept MaterialEntity until the term is accepted by TDWG, I'd stick with PreservedSpecimen.

[dustymc]

@Jegelewicz is my proposal accurate and acceptable and are we still good to go on this? (I could do this now, while I'm waiting for TACC to drag the test DB out of the black hole that it seems to have fallen into...)

[Jegelewicz]

@dustymc I say yes.

[campmlc]

I support.

On Tue, Aug 1, 2023, 5:22 PM Teresa Mayfield-Meyer @.***> wrote:

• [EXTERNAL]*

@dustymc https://github.com/dustymc I say yes.

— Reply to this email directly, view it on GitHub https://github.com/ArctosDB/arctos/issues/5459#issuecomment-1661233319, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADQ7JBDTALIRQXLI76SQK23XTGFURANCNFSM6AAAAAATV3ERPM . You are receiving this because you commented.Message ID: @.***>

[dustymc]

Fresh data:

temp_obs_citype(1).csv.zip

Done, BUT - I updated default_cat_item_type to machine observation for

guid_prefix

BYUObs:Mamm UTEPObs:Bird UTEPObs:Herb MVZObs:Herp BYUObs:Bird BYUObs:Herp Arctos:Entity MSBObs:Herp UMNH:Host UMNHObs:Mamm

@jtgiermakowski @mkoo @mvzhuang @Jegelewicz @byuherpetology

[mkoo]

@dustymc sorry for being late to the discussion-- please update MVZObs:Herp to human observation (that is the what we have for the others so would be consistent. We would reserve Machine Observation for camera traps and other unmanual-controlled (? what is the word I"m missing here!) media thanks!

[campmlc]

I second that request to change machine observation to human observation for Arctos:Entity. Not all entities are created using Arctos scripts to find related identifiers, and ultimately even with scripts it is people creating, entering, using, and determining that those identifiers are suitable for entity creation. This should not be confused with camera trap data.

[Jegelewicz]

As far as I can tell, all Arctos:Entity records are human observation already.

[dustymc]

@mkoo are you asking to update data or set the collection default?

[Jegelewicz]

I set the default for Arctos:Entity to human observation as well.

[Jegelewicz]

@dustymc are we done here?