ArctosDB / arctos

Arctos is a museum collections management system
https://arctos.database.museum
60 stars 13 forks source link

Code Table Request - new taxonomy source: United States Endangered Species #7023

Closed Jegelewicz closed 3 months ago

Jegelewicz commented 11 months ago

Goal

Describe what you're trying to accomplish. This is the only necessary step to start this process. The Committee is available to assist with all other steps. Please clearly indicate any uncertainty or desired guidance if you proceed beyond this step.

Provide information about the federal status of taxa.

Context

Describe why this new value is necessary and existing values are not.

As our current system seems to have failed, perhaps we can use the Federal ESA list:

https://www.ecfr.gov/current/title-50/chapter-I/subchapter-B/part-17/subpart-B/section-17.11#p-17.11(h)

to populate a taxonomy source as we recently did with IDNR status

See the developer tools on the site - can we download and transform (pretty sure I could use the HTML option to do this, but it would be clunky)? Get directly (that would be amazing)?

If we do this - we could update annually as the published CFR is updated annually. Just need a reminder.

Table

Code Tables are http://arctos.database.museum/info/ctDocumentation.cfm. Link to the specific table or value. This may involve multiple tables and will control datatype for Attributes. OtherID requests require BaseURL (and example) or explanation. Please ask for assistance if unsure.

https://arctos.database.museum/info/ctDocumentation.cfm?table=cttaxonomy_source

Proposed Value

Proposed new value. This should be clear and compatible with similar values in the relevant table and across Arctos.

United States Endangered Species

Proposed Definition

Clear, complete, non-collection-type-specific functional definition of the value. Avoid discipline-specific terminology if possible, include parenthetically if unavoidable.

Records the endangered/threatened status of species as listed by the United States. This source is expected to only include non-classification terms and must be manually updated each time “List of Endangered and Threatened Wildlife” is updated. This source is managed by the Arctos Taxonomy Committee.

Collection type

_Some code tables contain collection-type-specific values. collection_cde may be found from https://arctos.database.museum/home.cfm_

N/A

Attribute Extras

Attribute data type

If the request is for an attribute, what values will be allowed? free-text, categorical, or number+units depending upon the attribute (TBA)

N/A

Attribute controlled values

If the values are categorical (to be controlled by a code table), add a link to the appropriate code table. If a new table or set of values is needed, please elaborate.

N/A

Attribute units

if numerical values should be accompanied by units, provide a link to the appropriate units table.

N/A

Part preservation attribute affect on "tissueness"

if a new part preservation is requested, please add the affect it would have on "tissueness": No Influence, Allows, or Denies

N/A

Priority

Please describe the urgency and/or choose a priority-label to the right. You should expect a response within two working days, and may utilize Arctos Contacts if you feel response is lacking.

Setting as high - OGL has been using this already and all biological collections could benefit.

Example Data

Requests with clarifying sample data are generally much easier to understand and prioritize. Please attach or link to any representative data, in any form or format, which might help clarify the request.

Available for Public View

Most data are by default publicly available. Describe any necessary access restrictions.

Helpful Actions

@ArctosDB/arctos-code-table-administrators

Approval

All of the following must be checked before this may proceed.

_The How-To Document should be followed. Pay particular attention to terminology (with emphasis on consistency) and documentation (with emphasis on functionality). No person should act in multiple roles; the submitter cannot also serve as a Code Table Administrator, for example._

Rejection

If you believe this request should not proceed, explain why here. Suggest any changes that would make the change acceptable, alternate (usually existing) paths to the same goals, etc.

  1. Can a suitable solution be found here? If not, proceed to (2)
  2. Can a suitable solution be found by Code Table Committee discussion? If not, proceed to (3)
  3. Take the discussion to a monthly Arctos Working Group meeting for final resolution.

Implementation

Once all of the Approval Checklist is appropriately checked and there are no Rejection comments, or in special circumstances by decree of the Arctos Working Group, the change may be made.

Close this Issue.

DO NOT modify Arctos Authorities in any way before all points in this Issue have been fully addressed; data loss may result.

Special Exemptions

In very specific cases and by prior approval of The Committee, the approval process may be skipped, and implementation requirements may be slightly altered. Please note here if you are proceeding under one of these use cases.

  1. Adding an existing term to additional collection types may proceed immediately and without discussion, but doing so may also subject users to future cleanup efforts. If time allows, please review the term and definition as part of this step.
  2. The Committee may grant special access on particular tables to particular users. This should be exercised with great caution only after several smooth test cases, and generally limited to "taxonomy-like" data such as International Commission on Stratigraphy terminology.
Nicole-Ridgwell-NMMNHS commented 11 months ago

This seems like a good proposal. I am checking a box.

dustymc commented 10 months ago

The source is acceptable, checking boxes.

Any extra work from "we" needs addressed separately. (Tell them to send data to GlobalNames!)

Jegelewicz commented 7 months ago

I can create this source, but getting the information to populate it will be difficult

https://www.ecfr.gov/current/title-50/chapter-I/subchapter-B/part-17/subpart-B/section-17.11#p-17.11(h)

is pretty much useless as a transformation from HTML or PDF results in merged cells all over the place. Ideas appreciated.

image

image

Jegelewicz commented 7 months ago

and sometimes repetitive rows

image

campmlc commented 7 months ago

I would not have thought to look at the CFR for this. https://ecos.fws.gov/ecp0/reports/ad-hoc-species-report?kingdom=V&kingdom=I&status=E&status=T&status=EmE&status=EmT&status=EXPE&status=EXPN&status=SAE&status=SAT&mapstatus=3&fcrithab=on&fstatus=on&fspecrule=on&finvpop=on&fgroup=on&header=Listed+Animals

campmlc commented 7 months ago

Besides, that's not US only. That's probably IUCN redlist or something.

Jegelewicz commented 7 months ago

@campmlc thanks but even that resource has

image

why is all of this important information in such a terrible format?

Jegelewicz commented 7 months ago

The list provided at https://ecos.fws.gov/ecp0/reports/ad-hoc-species-report?kingdom=V&kingdom=I&status=E&status=T&status=EmE&status=EmT&status=EXPE&status=EXPN&status=SAE&status=SAT&mapstatus=3&fcrithab=on&fstatus=on&fspecrule=on&finvpop=on&fgroup=on&header=Listed+Animals is also not US only...

image

I actually think they are the same list, but they do seem to have a slightly different number of rows.

campmlc commented 7 months ago

We should ask @ccicero. I would trust USFWS on this before I trusted the US code of federal regulations. Certainly USFWS is whom I have to deal with for permits and import/export. The USFWS list is also available as csv download.

campmlc commented 7 months ago

But yes, the USFWS list is realy bad too - they don't have half the species we try to declare with anything like updated taxonomy.

Jegelewicz commented 7 months ago

OK. using the USFW list and checking for names in Arctos.

USFW Listed Animals.csv

waiting on mineral names to finish up first

Jegelewicz commented 3 months ago

names not found were added

Jegelewicz commented 3 months ago

This list adds a row for each status, so I did some manual manipulation to get a list of unique names with multiple statuses. I probably should have turned to R, but I don't have time now to relearn it.

Here is the file I worked on USFW Listed Animals.xlsx

Jegelewicz commented 3 months ago

Boxes are checked and the term has been added

Jegelewicz commented 3 months ago

I tried to load new classifications, but I am unable to add a no_class_term_type that isn't in the code table (I thought the code table was just a suggestion?) If a new term must be added, I can file a code table issue.

@dustymc please advise

dustymc commented 3 months ago

advise

Data please.

Jegelewicz commented 3 months ago

USFW Listed Animals.csv

dustymc commented 3 months ago

Assuming you're seeing the same error I'm seeing in test, there's nonprinting text (trailing whitespace, at least) in noclass_term_type_2.

Jegelewicz commented 3 months ago

DOH - thanks

Jegelewicz commented 3 months ago

The classifications are in the loader. So the source exists, but right now it isn't DOING anything.

File used to load classifications USFW Listed Animals.csv

Jegelewicz commented 3 months ago

If anyone wants some functionality from this source, they can request it.