Code Table Request - REMOVE endoparasite examination

dustymc commented 1 year ago

Initial Request

Goal: Describe what you're trying to accomplish. This is the only necessary step to start this process. The Committee is available to assist with all other steps. Please clearly indicate any uncertainty or desired guidance if you proceed beyond this step.

endoparasite examination https://arctos.database.museum/info/ctDocumentation.cfm?table=ctattribute_type#endoparasite_examination should be merged into some combination of

Migration path and value needs determined; I don't understand these data.

This will likely involve another code table request for a new value in https://arctos.database.museum/info/ctDocumentation.cfm?table=ctexamined_detected

Data:

temp_epe.csv.zip

Summary:


 guid_prefix | count 
-------------+-------
 ASNHC:Mamm  |    22
 DMNS:Bird   |  5976
 NMU:Mamm    |  1628
 DMNS:Herp   |     2
 DMNS:Mamm   |  3496
 UAM:Mamm    |     3
 MSB:Mamm    | 25377
 UWYMV:Herp  |     2
 MVZ:Herp    |    26
 MSB:Herp    |     5
 UNCG:Mamm   |   618
 MSB:Host    |  2995
 UMZM:Mamm   |    22
 UCM:Bird    |     1
 UMNH:Mamm   |   466
 MSB:Bird    |   106
 UWYMV:Bird  |    10
 UMZM:Bird   |    83
 UMNH:Bird   |     2
 MVZ:Bird    |   603

Users:

@ebraker @bryansmclean @mkoo @campmlc @ccicero @amgunderson @atrox10 @makaylaeasley @jtgiermakowski @keg34 @AdrienneRaniszewski @acdoll @jldunnum @jrdemboski @msbparasites @ewommack @kderieg322079 @catherpes @adhornsby

Context: Describe why this new value is necessary and existing values are not.

We seem to have two ways of doing the same thing.

Priority: Please describe the urgency and/or choose a priority-label to the right. You should expect a response within two working days, and may utilize Arctos Contacts if you feel response is lacking.

Code table cleanup should always be highest priority.

@ArctosDB/arctos-code-table-administrators

Approval

All of the following must be checked before this may proceed.

_The How-To Document should be followed. Pay particular attention to terminology (with emphasis on consistency) and documentation (with emphasis on functionality)._

[x] Code Table Administrator[1] - check and initial, comment, or thumbs-up to indicate that the request complies with the how-to documentation and has your approval
[ ] Code Table Administrator[2] - check and initial, comment, or thumbs-up to indicate that the request complies with the how-to documentation and has your approval
[x] DBA - The request is functionally acceptable. The term is not a functional duplicate, and is compatible with existing data and code.
[x] DBA - Appropriate code or handlers are in place as necessary. (ID_References, Media Relationships, Encumbrances, etc. require particular attention)

Rejection

If you believe this request should not proceed, explain why here. Suggest any changes that would make the change acceptable, alternate (usually existing) paths to the same goals, etc.

Can a suitable solution be found here? If not, proceed to (2)
Can a suitable solution be found by Code Table Committee discussion? If not, proceed to (3)
Take the discussion to a monthly Arctos Working Group meeting for final resolution.

Implementation

Once all of the Approval Checklist is appropriately checked and there are no Rejection comments, or in special circumstances by decree of the Arctos Working Group, the change may be made.

Review everything one last time. Ensure the How-To has been followed. Ensure all checks have been made by appropriate personnel.

Make changes as described above. Ensure the URL of this Issue is included in the definition.

Close this Issue.

DO NOT modify Arctos Authorities in any way before all points in this Issue have been fully addressed; data loss may result.

Special Exemptions

In very specific cases and by prior approval of The Committee, the approval process may be skipped, and implementation requirements may be slightly altered. Please note here if you are proceeding under one of these use cases.

Adding an existing term to additional collection types may proceed immediately and without discussion, but doing so may also subject users to future cleanup efforts. If time allows, please review the term and definition as part of this step.
The Committee may grant special access on particular tables to particular users. This should be exercised with great caution only after several smooth test cases, and generally limited to "taxonomy-like" data such as International Commission on Stratigraphy terminology.

campmlc commented 1 year ago

I support this for MSB collections. However, we need first to add "endpparasites" as an option in https://arctos.database.museum/info/ctDocumentation.cfm?table=ctexamined_detected

And the examined for attribute will need to be expanded to include the following collections: Bird Herp Inv Teach

I'm assuming any date, determiner, method and remarks will also get transferred.

dustymc commented 1 year ago

first to add "endpparasites" as an option

Please file and issue or provide a definition and we'll do it for you.

campmlc commented 1 year ago

Parasitism is s form of symbiosis in which one organism (called parasite) benefits at the expense of another organism usually of different species (called the host). This host-parasite association may eventuate in the injury of the host. Parasites may be grouped into ectoparasites and endoparasites. Parasites that live outside the host are called ectoparasites whereas those that live inside the host are called endoparasites.

Endoparasites are of two forms: intercellular parasites and intracellular parasites.

Intercellular parasites are those that inhabit the spaces of the body of the host. Examples of intercellular parasites are nematodes, tapeworms, and other helminths. Helminths live in the gut of their hosts. Intracellular parasites are endoparasites that live within the cell of the host. Examples of intracellular parasites are the protozoan Plasmodium, the causative agent of malaria. They thrive inside the cells of their human host. Plasmodium species have different stages in their life cycle. Within the definitive host (human), the sporozoite stage of Plasmodium species occurs within the liver cells where the sporozoite gives rise to a merozoite or to a hypnozoite, which then infects the red blood cell of the host.

Jegelewicz commented 1 year ago

@campmlc please file separate code table issues for each term that is needed.

dustymc commented 1 year ago

Anyone can get the wikipedia copypasta, can we cay something FUNCTIONAL about what this term actually means for the material that a user might find in Arctos?

campmlc commented 1 year ago

For this particular definition, we need to include both intercellular and intracellular, as I'm pretty sure we had had to use for both since this is currently the only available attribute. And I really don't have time right now to file separate issues. If that is required this is going to have to be on hold until I do. Can't we just post a google sheet in one issue and agree to it? There are too many issues to separate out each one. Every examined-detected value needs three !

campmlc commented 1 year ago

What I posted is what is functionally in use in Arctos. This is a very vague term as it is currently used because we had no other option - it had to stand in for everything. Now we are going to have other more specific options, and eventually maybe we can parse these legacy values out. But that is not currently the case.

acdoll commented 1 year ago

Wait - I feel like getting rid of this will lose some of the data we have been diligently collecting since this attribute was created. Moving the 'endoparasite examination'=yes to 'examined for'=endoparasite makes sense; but what happens to all of the 'endoparasite examination'=no? That is important data: Did we not detect endoparasites because we looked and they weren't there, or because we didn't look?

dustymc commented 1 year ago

@campmlc I'm not sure what you're willing to support, I haven't proposed anything beyond somehow getting rid of this.

@acdoll see original comment, I'm not entirely convinced that anyone understands any of this mess. A no on this particular attribute (particularly in context of a record not having parasites) implies - ???? I don't know, but hopefully it makes sense to whoever's been using it, and hopefully ya'll will tell me how we can get it into the recently-developed system that we've been discussing (https://github.com/ArctosDB/arctos/issues/5688), which hopefully does make sense to everyone!

campmlc commented 1 year ago

I tried valiantly to the point of alienating everyone to add "not examined" as a fourth attribute, but was overruled. In the interest of simplicity going forward, perhaps we leave endoparasite examination = no as it's own attribute, and migrate the other yes values.

acdoll commented 1 year ago

I guess a no implies that this individual should be excluded from any parasite prevalence analysis (maybe it was chock-full of parasites but we were to busy to check or collect). Such an analysis should probably only use records with confirmed examinations, but that doesn't work for historical specimens. I could imagine someone just assuming everything was checked unless otherwise noted (such as with 'endoparasite examination'=no). Really, the yes/no was a way of being explicit about what is happening in the lab currently. That is, as opposed to a yes only option: Did someone actually not check for parasites (no) or did they just forget to write down that they did check for them (blank)?

dustymc commented 1 year ago

excluded from any parasite prevalence analysis

... which is identical to no information at all.

someone just assuming everything was checked

I don't see how this could ever be a valid assumption (and I think any serious research would have to exclude anything without method, so even if that assumption could somehow be justified it would still be a dead end).

happening in the lab

Perhaps a job for https://arctos.database.museum/info/ctDocumentation.cfm?table=ctflags? (Whatever the details, the idea that we shouldn't mix procedural and Research Grade information comes up from time to time.)

campmlc commented 1 year ago

Regardless, we need a place to put this explicitly documented negative information, and right now, that would be what, collection object remarks? I agree that throwing away good data (as good as we were allowed to capture at the time these attributes were entered) is a shame. We lack a lot of parasite data in host records because of the 10 column limit on attributes, not because we don't have the data. Someone went to a lot of trouble to say that a host was not examined for endoparasites, and now we have no place to put those data. MSB has captured these values, and apparently so has Denver. Let's not lose them.

On Fri, May 5, 2023, 3:27 PM dustymc @.***> wrote:

[EXTERNAL]*

excluded from any parasite prevalence analysis

... which is identical to no information at all.

someone just assuming everything was checked

I don't see how this could ever be a valid assumption (and I think any serious research would have to exclude anything without method, so even if that assumption could somehow be justified it would still be a dead end).

happening in the lab

Perhaps a job for https://arctos.database.museum/info/ctDocumentation.cfm?table=ctflags? (Whatever the details, the idea that we shouldn't mix procedural and Research Grade information comes up from time to time.)

— Reply to this email directly, view it on GitHub https://github.com/ArctosDB/arctos/issues/6222#issuecomment-1536798952, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADQ7JBAPMDM4JA4LB6NDG2TXEVWFLANCNFSM6AAAAAAXUQZJHM . You are receiving this because you were mentioned.Message ID: @.***>

campmlc commented 1 year ago

Why don't we go back to one of the original proposal, which is to add the value "not examined" to the examined_detected code table. Then for these legacy records, puy in methods the standardize text "not examined for endoparasites" with the original date and determiner and remarks copied over. This could be the same protocol for parasite examination = no and ectoparasite examination = no. This way, we can capture the explicit "not examined" values used at MSB and Denver and elsewhere. The other alternative is to create a fourth attribute of "not examined for" and use the same code table. @acdoll

campmlc commented 1 year ago

Personally, I think it makes the most sense to add the fourth attribute. This is what I advocated for previously. We are converting data in two attributes, each with yes/no values. Logically, these should be replaced by four attributes, as follows: Original attributes Values Examined for endoparasites yes, no Endoparasites detected yes, no

Replace with Examined for: endoparasites Not Examined for: endoparasites Detected endoparasites Not Detected endoparasites

Ditto for existing examined for parasites yes/no, parasites detected yes/no, examined for ectoparasites yes/no, ectoparasites detected yes/no attributes. The date and determiner and method when available will be copied over from the original. I am happy to update all MSB values with a methods field if given a csv of all existing values for these three attributes.

jldunnum commented 1 year ago

I support having a "not examined for" attribute. That is very different from a negative value in "examined for" and we do have these data so we should capture them.

dustymc commented 1 year ago

I just saw an error message involving this, the existence of this is acting as a denormalizer and causing unnecessary work, it needs pushed to completion in some way. @mkoo help?

campmlc commented 1 year ago

This can move forward if we can create the not examined attribute as requested in #6219. Then the "endoparasite examination" "yes/no" values and all remarks and metadata should be migratated to the new attributes. Old "endoparasite examination" = "yes" becomes "examined for" value "endoparasites". Old endoparasite examination= no becomes "not examined for" = "endoparasites". If I can get some time this weekend while I'm in the field, I can add methods to the MSB attributes. The date should be the collecting event date for all MSB records.

dustymc commented 1 year ago

I believe this was sufficiently approved by @ArctosDB/arctos-code-table-administrators and I would like to begin work on it TOMORROW.

@campmlc @ccicero @jldunnum please confirm and check some boxes.

campmlc commented 1 year ago

To be converted into examined for and not examined for attributes. Approved.

jldunnum commented 1 year ago

I support but don’t think I have approval power

Get Outlook for iOShttps://aka.ms/o0ukef

From: Mariel Campbell @.> Sent: Thursday, May 18, 2023 5:14:45 PM To: ArctosDB/arctos @.> Cc: Jonathan Dunnum @.>; Mention @.> Subject: Re: [ArctosDB/arctos] Code Table Request - REMOVE endoparasite examination (Issue #6222)

[EXTERNAL]

To be converted into examined for and not examined for attributes. Approved.

— Reply to this email directly, view it on GitHubhttps://github.com/ArctosDB/arctos/issues/6222#issuecomment-1553770510, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AED2PA4ACPIF65CN37O2EI3XG2UOLANCNFSM6AAAAAAXUQZJHM. You are receiving this because you were mentioned.Message ID: @.***>

dustymc commented 1 year ago

don’t think I have approval power

You showed up and helped us get through some stuff, AFAIK that's exactly what's required to wield the superpowers.

jldunnum commented 1 year ago

Wow if that's all it takes I should how up to more things in life!

Jonathan L. Dunnum Ph.D. (he, him, his) Senior Collection Manager Division of Mammals, Museum of Southwestern Biology University of New Mexico Albuquerque, NM 87131 (505) 277-9262 Fax (505) 277-1351

Chair, Systematic Collections Committee, American Society of Mammalogists Latin American Fellowship Committee, ASM

MSB Mammals website: http://www.msb.unm.edu/mammals/index.html Facebook: http://www.facebook.com/MSBDivisionofMammals

Shipping Address: Museum of Southwestern Biology Division of Mammals University of New Mexico CERIA Bldg 83, Room 204 Albuquerque, NM 87131

From: dustymc @.> Sent: Friday, May 19, 2023 8:45 AM To: ArctosDB/arctos @.> Cc: Jonathan Dunnum @.>; Mention @.> Subject: Re: [ArctosDB/arctos] Code Table Request - REMOVE endoparasite examination (Issue #6222)

[EXTERNAL]

don’t think I have approval power

You showed up and helped us get through some stuff, AFAIK that's exactly what's required to wield the superpowers.

— Reply to this email directly, view it on GitHubhttps://github.com/ArctosDB/arctos/issues/6222#issuecomment-1554701016, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AED2PAYPGRQRX5IXK6AW6ITXG6BPBANCNFSM6AAAAAAXUQZJHM. You are receiving this because you were mentioned.Message ID: @.***>

acdoll commented 1 year ago

jldunnum commented 1 year ago

[like] Jonathan Dunnum reacted to your message:

From: Andrew Doll @.> Sent: Friday, May 19, 2023 3:46:39 PM To: ArctosDB/arctos @.> Cc: Jonathan Dunnum @.>; Mention @.> Subject: Re: [ArctosDB/arctos] Code Table Request - REMOVE endoparasite examination (Issue #6222)

[EXTERNAL]

[image]https://user-images.githubusercontent.com/12507297/239577020-6c5b98a6-2c65-4407-af9a-75dc3e5f72e2.png

— Reply to this email directly, view it on GitHubhttps://github.com/ArctosDB/arctos/issues/6222#issuecomment-1554775420, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AED2PA5ZTTFPZRYHEH7RZ43XG6IV7ANCNFSM6AAAAAAXUQZJHM. You are receiving this because you were mentioned.Message ID: @.***>

msbparasites commented 1 year ago

I caught up a bit today on all this, but have also talked with Mariel off and on. As someone who uses "examined for" (with controlled dropdown) all the time and "not examined" I think that is giving it an upgrade of more information than what we have right now. At some point being able to select what was examined for would be useful, right now I have to put that in comments, but that is for the HOST records I create, I have no control over mammals or birds and how they display what was examined, as many of the rodents they examine only the GI tract is examined, not brain or eyes and no blood smear for malaria, for example. Thanks for cleaning this up.

dustymc commented 1 year ago

GI tract is examined, not brain or eyes and no blood

Almost like the data are crying out for parts....

Fresh data:

temp_noexamfor_endo.csv.zip temp_examfor_endo.csv.zip temp_epe(1).csv.zip

Done.

ArctosDB / arctos