ArctosDB / arctos

Arctos is a museum collections management system
https://arctos.database.museum
60 stars 13 forks source link

Help needed in Arctos: WoRMS taxonomy! #8130

Open happiah-madson opened 1 month ago

happiah-madson commented 1 month ago

So I was looking through our collections taxonomy and discovered that some fish have class Actinopterygii and some fish have class Actinopteri and if I look at WoRMS these fish should really be class Teleostei. Because taxonomy is shared, I'm hesitant to force a WoRMS via Arctos refresh (also, how would I do this for forcing "all fish"). But clicking refresh for every taxa that is a Teleost seems...not efficient?

What are relevant pages in Arctos

https://arctos.database.museum/name/Secutor%20insidiator#WoRMSviaArctos

This is an example of a species that we have where WoRMS says teleostei and WoRMS via Arctos says Actinopteri and I want to update WoRMS via Arctos but I don't want to mess things up.

@falco-rk @dustymc @sharpphyl @mkoo @genevieve-anderegg

dustymc commented 1 month ago

Anyone using service-sources data has to be prepared for those data to follow the service at any time, wherever weird place it might go. That should be (kinda isn't) perfectly clear from the definition.

refresh for every taxa

WoRMS: Request Refresh

all fish

We're, like, all fish, man! https://arctos.database.museum/name/Homo%20sapiens#OpenTreeofLife

happiah-madson commented 1 month ago

Okay, let's pretend I'm very dense (...). Do I type "Teleostei" in the Request Refresh field?

dustymc commented 1 month ago

very dense

Sorry, that's mine!

I'm hoping @sharpphyl - who's actually used this thing - will show up and clarify. In theory: Yea, you'd feed the form something like subclass + Teleostei and it'll flag everything to be refreshed (which might take a couple days). In reality: I'm not sure, possibly that'll just melt something important....

Once things are better understood, I'd be happy to update the form to include any documentation that might be supplied here.

happiah-madson commented 1 month ago

Great, I will wait for the wisdom of @sharpphyl

genevieve-anderegg commented 1 month ago

Okay, let's pretend I'm very dense (...). Do I type "Teleostei" in the Request Refresh field?

Yep! Prepare for it to take awhile to work through everything (hours-days).

Anyone using service-sources data has to be prepared for those data to follow the service at any time, wherever weird place it might go. That should be (kinda isn't) perfectly clear from the definition.

Yes, so if we have specimens where the taxonomy hasn't been updated for us yet, it will change it for us too. But, we don't have any fish records that use the Worms via Arctos source in our collection, and like Dusty is saying, it's a benefit of using the shared source BUT that also means we have to keep pretty on top of our taxonomy. Thanks for the heads up, go ahead!

I'm hoping @sharpphyl - who's actually used this thing - will show up and clarify.

I hope I'm a suitable stand in ;)

happiah-madson commented 1 month ago
Screen Shot 2024-09-23 at 1 31 26 PM

I feel I am doing something horribly incorrectly. (Do I realize that updating all chordata is insane? yes. But all of my other queries were returning 0 records.) Help?

dustymc commented 1 month ago

Oh thanks @genevieve-anderegg didn't realize you use this too.

insane

The DB agrees! (That might work sometimes - depending on what else is going on at the moment - so there's probably no sort of 'this rank' or 'that many records' limit guidance.)

This worked for me:

Screenshot 2024-09-23 at 11 23 15

and so did the next click

Screenshot 2024-09-23 at 11 25 07

you can check in with...

Screenshot 2024-09-23 at 11 25 18

https://arctos.database.museum/tools/requestWormsRefresh.cfm?action=checkStatus

Screenshot 2024-09-23 at 11 26 31

and hopefully i didn't just plug the toobs with something you don't care about....

happiah-madson commented 1 month ago

Okay, so the problem I was clearly having was ascribing Teleostei to the wrong rank (I put it in as a class b/c WoRMS says that it's a class now....). How do I know which rank to give it? (How did you know to use subclass?)

dustymc commented 1 month ago

The query is using current Arctos data

https://arctos.database.museum/name/Secutor%20insidiator#WoRMSviaArctos

Screenshot 2024-09-23 at 11 38 02
happiah-madson commented 1 month ago

Oooooooo v good. Helpful. Thank you.

happiah-madson commented 1 month ago

I continue to struggle. @dustymc your update did things! I can see that some records have been updated and have the Class Teleostei. However, there are still things that are Actinopterygii. For example:

Screen Shot 2024-09-25 at 1 51 11 PM

which according to WoRMS should be Teleostei (https://www.marinespecies.org/aphia.php?p=taxdetails&id=127426). My understanding is that to do this I should go to https://arctos.database.museum/tools/requestWormsRefresh.cfm and put in:

term type (exact case-sensitive): class term (case insensitive, wildcard is %): Actinopterygii

But when I do that, I get this:

Screen Shot 2024-09-25 at 1 54 23 PM

Is this just because Actinopterygii is B I G? or is there something that I am doing wrong?

dustymc commented 1 month ago

just because Actinopterygii is B I G?

Yup.

  1. Maybe try back later, possible things are just busy (but probably not in this case)
  2. Chop it up somehow (I see this one doesn't have subclass so that's not handy)
  3. Ping me and I'll figure it out

Am I missing anything @genevieve-anderegg @sharpphyl ?

genevieve-anderegg commented 1 month ago

This has happened to us too, I try and then chop it up like you suggested. Hard if the next level of taxonomy has lot of different terms

happiah-madson commented 1 month ago

If you're not missing anything, can I assume this counts as

Ping me and I'll figure it out

happiah-madson commented 1 month ago

This has happened to us too, I try and then chop it up like you suggested. Hard if the next level of taxonomy has lot of different terms

😭😭😭😭😭😭😭😭😭😭😭😭😭

dustymc commented 1 month ago

assume this counts

Sure!

INSERT 0 75243

a bunch of those probably got refreshed a couple days ago, might be useful to mention that (whatever it is - order of operations??) in the documentation if possible

sharpphyl commented 1 month ago

I think Genna has been a more-than-adequate stand in. Currently Arctos is churning away on your request and I would think it could be a week or more before it's done - depending on what else is going on - and until it's done churning, you'll have inconsistencies in the classifications. In our case, we manually refreshed the few that hadn't gotten to the front of the queue yet if we needed them.

Screenshot 2024-09-30 at 8 18 03 PM

You will want to ask it to try again on anything that fails.

Dusty can tell us if there's an actual limit that triggers errors, but I unusually was able to stay around 10,000 - 25,000 entries with mollusca but that may not work for your fish.

As more people use WoRMS for more phyla, this may become an issue if we all refresh at the same time. Mollusca is one of the largest with over 250,000 names and I've been trying to refresh all the classes we have each year. Not all of them are in Arctos, so we're refreshing way fewer than that. Right now, it's first come, first serve, so have at it!