ArctosDB / arctos

Arctos is a museum collections management system
https://arctos.database.museum
60 stars 13 forks source link

Code Table Request - edit cultures #3457

Closed AJLinn closed 2 years ago

AJLinn commented 3 years ago

Goal In an attempt to be more consistent with changing trends in self-identification of cultures, I think it's time for us to look at some of the terms in our culture code table for Alaska and the north.

Context The existing CT uses the terms "Eskimo" and "Athabascan" as collective terms to reference items of certain generic characteristics that can't be narrowed down to more specific cultural attributes. It is likely time for us to make a shift to "Inuit" and "Athabascan/Dene" as the collective terms when we can't be specific.

Table https://arctos.database.museum/info/ctDocumentation.cfm?table=ctculture

Value All references to "Eskimo" be shifted to "Inuit" All references to "Athabascan" be shifted to "Athabascan/Dene"

Definition I'll work on doing a one to one shift for all affected terms.

Collection type EH, Art, Arc

[not sure if any of the below apply to this request] Attribute data type "Attributes" may apply to catalog records, parts, localities, and collecting events. You must specify a datatype (free-text, categorical, or number+units) if this request involves attributes.

Attribute value For categorical attributes, code table controlling value

Attribute units For number+units attributes, code table controlling units

Part tissue flag For new parts, is the part a tissue?

Other ID BaseURL For OtherIDs, provide the following or explain why the unresolvable ID type is necessary:

  1. "Base URL" with which to prepend entered values, and
  2. A functional example URL, which should consist of the base URL provided in (1) plus a relevant value._

ID_References If the request involves https://arctos.database.museum/info/ctDocumentation.cfm?table=ctid_references, the changes must be coordinated with the DBA team for notifications to function._

Priority Medium

dustymc commented 3 years ago

Any path to this involves first creating new terms and definitions - it's not possible to just update.

You can use the attribute unloader to remove existing data, then the loader to create new, or I can make those updates for you.

Making sure we don't have two ways of saying the same thing is critical - here's who's using.

select guid_prefix,  count(*) c
from
attributes
inner join cataloged_item on attributes.collection_object_id=cataloged_item.collection_object_id
inner join collection on cataloged_item.collection_id=collection.collection_id
where attribute_value in ('Eskimo','Athabascan')
group by guid_prefix
order by guid_prefix
;

 guid_prefix |  c   
-------------+------
 CHAS:EH     |    1
 UAM:Arc     | 5023
 UAM:Art     |    7
 UAM:EH      | 2368
AJLinn commented 3 years ago

Here's the google sheet I've started where I'll be making updates. Tips for how this will work better with our unloader/loader will be appreciated.

Jegelewicz commented 3 years ago

Tips for how this will work better with our unloader/loader will be appreciated.

@AJLinn go here to get an attribute UNload template: https://arctos.database.museum/tools/BulkUnLoadAttribute.cfm?action=ld

and here to get an attribute load template: https://arctos.database.museum/tools/BulkloadAttributes.cfm?action=ld

These both have been re-done using the new batch tools template and they work the same way. Create your csv and load it. If you create the csv with "autoload" in the status column, the records will begin to load automatically once you load the csv. If you leave status blank or put anything besides "autoload" in that column, the records will sit in the tool until you change the status to "autoload". If there is a problem with any of the records, the error will appear in the status column and the records will remain in the tool. You can download a csv of these errors, correct them and re-load.

Let me know if you have any questions about the tools.

Jegelewicz commented 3 years ago

Here's the google sheet I've started where I'll be making updates.

These are changes that will need to be made in the code table - once you are finished, let us know and we can review before making an adds to the code table. Deletions from the code table cannot happen until the terms are no longer in use.

AJLinn commented 3 years ago

Once my team of advisors help me determine what changes need to happen, I'll schedule time to chat with you about the steps to implementation.

One question: if there are comments in the attribute fields (e.g., remarks, determined method) will those go away if we update the value in those attributes? Those are important for us to maintain regarding who made the attribution and what they were based on. These updates that I'm going to be recommending are not changing the attribute, just the words used to describe the same people.

dustymc commented 3 years ago

Create your csv

Which can be done via....

Screen Shot 2021-02-23 at 8 03 45 AM

FWIW I tested the initial build by unloading and reloading all attributes for some random catalog numbers - it's 4 or 5 clicks (I didn't edit the CSV) and I think pretty easy/intuitive, but we should probably have a how-to anyway (if you'd like to volunteer......).

Deletions from the code table cannot happen until the terms are no longer in use

I just want to stress that again - please don't let this get hung up halfway, we REALLY need to avoid multiple ways of saying the same thing.

Jegelewicz commented 3 years ago

but we should probably have a how-to anyway (if you'd like to volunteer......).

A video tutorial as you do this would be FANTASTIC!

Jegelewicz commented 3 years ago

@droberts49 can you look at the one CHAS item for us so that doesn't prove to be the hang-up?

AJLinn commented 3 years ago

I'd like to follow up on this update. Should I schedule a meeting with relevant groups or should this be brought to the Issues or maybe the code table committee meeting? Please advise.

Jegelewicz commented 3 years ago

@AJLinn Whoever is managing UAM:Arc, @marecaguthrie and @droberts49 are the interested parties. If they agree, we need definitions for the new attributes:

Inuit Athabascan/Dene

so that they can be added to the code table before any unload/load process can happen.

AJLinn commented 3 years ago

@sjshirar do you have any issues with me implementing these name changes Scott? Take a look at the Google sheet for the changes. @Jegelewicz I'll indicate in the spreadsheet the definitions for all the ones that are changing (most noticeably all those that were "Eskimo, XXXX" are now just the "XXXX" part, so those will be seen as "new", is that right?

Thanks to everyone who will be involved in making this change happen. It's an important one.

dustymc commented 3 years ago

@AJLinn are you sure about the UTF characters? It's no problem for Arctos, but eg Excel probably has about 5000 ways to mangle Sámi which could complicate things like data entry.

Also, questions. Athabascan/Dene, Hän for example

  1. the doc is a link to the language (https://en.wikipedia.org/wiki/H%C3%A4n_language), https://en.wikipedia.org/wiki/H%C3%A4n seems more appropriate (even though it just redirects to the language at the moment - but that could change). Was that intentional?
  2. What's up with the Athabascan/Dene, ... bits; why not just eg Hän? Any ambiguity can be addressed by the documentation, if that's the concern. Maybe it's a way to get at 'find all Athabascan' (which does seem reasonable)?

And this seems to be well past the point where using bulk tools is efficient; let me know when the new values exist and the mapping is finalized and I can SQL update.

AJLinn commented 3 years ago

Thanks for your questions Dusty. My cultural advisors have indicated that the diacritics are important to the members of those cultures to properly cite their cultural designations. That said, if you think our data entry issues will create errors, we can use the diacritics in the definition rather than in the values - I'm guessing that's pretty standard. Seems to mess up copying URLs also (https://en.wikipedia.org/wiki/H%C3%A4n case in point...)

I fixed the link for Hän culture to the correct wiki (must not have existed when we did the initial documentation!)

What's up with the Athabascan/Dene, ... bits; why not just eg Hän? Any ambiguity can be addressed by the documentation, if that's the concern. Maybe it's a way to get at 'find all Athabascan' (which does seem reasonable)?

Yes, this is exactly the problem I'm facing. But...you're right: to be consistent with our updates to the formerly "Eskimo" groups we should drop it.

I'll keep massaging the entries, checking for accuracy, and request one final review from my research affiliates in my department before we do any surgery. Just wanted to get the ball in motion again. Thanks for the feedback.

dustymc commented 3 years ago

data entry issues will create errors

Your call. If you're comfortable with unicode tools, you probably won't notice. If you regularly use anything Microsoft, you'll notice....

Yes the current standard requires encoding non-ASCII characters in URLs - I keep thinking that'll change but here we are!

It would be a bit more maintenance, but you could just load multiple attributes to support discovery:

are all accurate if not terribly precise, and would make "actual value" records discoverable by eg giant group. I'm not sure that's quite a recommendation, but it's an option.

sjshirar commented 3 years ago

Angie, no issues here with any of these changes.

On Wed, Aug 25, 2021 at 2:48 PM Angela Linn @.***> wrote:

@sjshirar https://github.com/sjshirar do you have any issues with me implementing these name changes Scott? Take a look at the Google sheet for the changes. @Jegelewicz https://github.com/Jegelewicz I'll indicate in the spreadsheet the definitions for all the ones that are changing (most noticeably all those that were "Eskimo, XXXX" are now just the "XXXX" part, so those will be seen as "new", is that right?

Thanks to everyone who will be involved in making this change happen. It's an important one.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ArctosDB/arctos/issues/3457#issuecomment-905926280, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGWDEEIPGNVVZJKIEOBMAULT6VXKJANCNFSM4YBS22VQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

-- Scott Shirar Archaeology Collection Manager University of Alaska Museum of the North 907-474-6943

AJLinn commented 3 years ago

It would be a bit more maintenance, but you could just load multiple attributes to support discovery:

That's what I was thinking too - so do the "Han" for attribute one, then "Athabascan/Dene" for a second. Probably the only way we can accurately get at the info we regularly are asked for.

@marecaguthrie Are you good with the changes I am proposing? We'll need to update your 7 entries.

marecaguthrie commented 3 years ago

I'm excited about these changes and 100% supportive. Thanks for all your work on this.

On Wed, Sep 1, 2021 at 4:24 PM Angela Linn @.***> wrote:

It would be a bit more maintenance, but you could just load multiple attributes to support discovery:

That's what I was thinking too - so do the "Han" for attribute one, then "Athabascan/Dene" for a second. Probably the only way we can accurately get at the info we regularly are asked for.

@marecaguthrie https://github.com/marecaguthrie Are you good with the changes I am proposing? We'll need to update your 7 entries.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ArctosDB/arctos/issues/3457#issuecomment-910944592, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJKSRR4DUSDMVQEB5EF3NG3T7275JANCNFSM4YBS22VQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

-- Mareca Guthrie (she/her/hers) Curator of Fine Arts & Associate Professor of Art University of Alaska Museum of the North 1962 Yukon Drive P.O. Box 756960 Fairbanks, AK 99775-6960 @.***

University of Alaska Museum of the North: www.uaf.edu/museum UAF Art Department: https://www.uaf.edu/art/ https://www.uaf.edu/art/

At UAF, we acknowledge the Alaska Native nations upon whose ancestral and unceded lands our six campuses reside. Here in Fairbanks, our Troth Yeddha' Campus is located on the lands of the Dené people of the Tanana River.

AJLinn commented 2 years ago

@Jegelewicz and @dustymc are we ready to have me move forward on this? Is there an upcoming office hours when we might schedule a tutorial on how to make the changes? Or can I request a meeting with one or both of you to go thru the steps? I have edit access to all three UAM cultural collections if @sjshirar and @marecaguthrie trust me to implement the changes for their records as well? Thanks!

dustymc commented 2 years ago
  1. Create the new terms
  2. Send me CSV containing old and new values

Is https://docs.google.com/spreadsheets/d/1XkwfjYAT_5ya37RdAPtr9xUwADJ9BG-_2AUaol9XrDU/edit#gid=0 current? I think it contains everything I'd nee.

Jegelewicz commented 2 years ago

@AJLinn let me know if you need any help with no. 1. above.

AJLinn commented 2 years ago

Is https://docs.google.com/spreadsheets/d/1XkwfjYAT_5ya37RdAPtr9xUwADJ9BG-_2AUaol9XrDU/edit#gid=0 current? I think it contains everything I'd nee.

Yup, looks good. Did we decide to cut the diacritics because of possible problems with Excel? I think we'll be okay as long as they are in the definition - maybe if we have to cut the diacritic we can add to the documentation "Preferred Hän" for example?

I defer to @dustymc for the functionality. The cultural experts say keep them if at all possible. Thanks!

dustymc commented 2 years ago

Arctos does not care - a UTF character is a UTF character. If whatever you use to make CSV can export them intact and properly encoded, then goferit.

Jegelewicz commented 2 years ago

@AJLinn if you save as from Excel and choose CSV UTF-8 (Comma delimited) (*.csv) they should work.

AJLinn commented 2 years ago

@dustymc

  1. Do you need all the comments removed?
  2. How about things that are not changing? Just bring the terms and definitions over as-is or leave the cells empty?
dustymc commented 2 years ago

Na, I can just create new CULTURE def new documentation when it doesn't exist. Just say "go"....

AJLinn commented 2 years ago

Go! ...(Unless you need me to download & save as CSV as Teresa mentioned above?)

dustymc commented 2 years ago

done