ArctosDB / arctos

Arctos is a museum collections management system
https://arctos.database.museum
60 stars 13 forks source link

Feature Request - basisOfRecord / ctcataloged_item_type - the saga continues #7687

Closed dustymc closed 6 months ago

dustymc commented 6 months ago

Background: https://github.com/ArctosDB/arctos/issues/6730

Is your feature request related to a problem? Please describe.

1 line(s) have basisOfRecord that does not match the Darwin Core Type Vocabulary (please note comparisons are case insensitive)

Noncompliant vocabulary (MaterialEntity) breaks GBIF. I don't have a great way to deal with it, but I can do something not-great if I must. @dbloom should absolutely not be stuck dealing with this.

Describe what you're trying to accomplish

Not deal with this mess any more.

Describe the solution you'd like

Someone please tell me how I can avoid dealing with this mess ever again.

Describe alternatives you've considered

Silently and cryptically (because I have no idea how else I could do it) exclude noncompliant records from DWC.

Additional context

Summary data, but I think many of these don't publish DWC. DMNS Marine Inverts and Trinity College Paleo are the collections that melted Dave's thingee.


                      collectionid                       | count  
---------------------------------------------------------+--------
 https://arctos.database.museum/collection/ALMNH:EH      |   1137
 https://arctos.database.museum/collection/CHAS:Art      |      6
 https://arctos.database.museum/collection/CHAS:EH       |     61
 https://arctos.database.museum/collection/CHAS:Teach    |     10
 https://arctos.database.museum/collection/DMNS:Inv      |      1
 https://arctos.database.museum/collection/MVZ:Arch      |    244
 https://arctos.database.museum/collection/NHSM:Arc      |   2030
 https://arctos.database.museum/collection/OGL:Genomic   |     73
 https://arctos.database.museum/collection/TCDGM:Mineral |      1
 https://arctos.database.museum/collection/TCDGM:Paleo   |      1
 https://arctos.database.museum/collection/UAM:Arc       | 734808
 https://arctos.database.museum/collection/UAM:Art       |   5511
 https://arctos.database.museum/collection/UAM:EH        |  38511
 https://arctos.database.museum/collection/UNM:MET       |   5422

Priority

critical

dbloom commented 6 months ago

Yeah, it sucks to have a, um, melted thingy.

In fact when it works properly, I don't get errors about collections such as DMNS Marine Inv and Trinity College Paleo.

This is the current vocab for dwc:basisOfRecord for publication via the IPT and into GBIF (IPT or not). https://dwc.tdwg.org/terms/#dwc:basisOfRecord

If you are actually using "MaterialEntity" correctly, per the link above, then please let me know so I can log an issue/inquiry with GBIF. You might also need to double check spelling, look for correct capitalization, no additional spaces, etc... It certainly makes sense to use this basisOfRecord for these collections, but before I go to GBIF it would be good to know that the issue is not on the Arctos side.

Currently, the ability to search by basisOfRecord in the GBIF portal is limited to these terms.

Screen Shot 2024-04-18 at 4 12 08 PM

That doesn't mean GBIF is correct, but until addressed, these two collections are not likely to update to GBIF.

Let me know.

dustymc commented 6 months ago

actually using "MaterialEntity" correctly

Our intent should align: https://arctos.database.museum/info/ctDocumentation.cfm?table=ctcataloged_item_type.

Authorities are always enforced:


arctosprod@arctos>> \d cataloged_item;
                                  Table "core.cataloged_item"
        Column        |            Type             | Collation | Nullable |      Default      
----------------------+-----------------------------+-----------+----------+-------------------
 collection_object_id | integer                     |           | not null | 
 cataloged_item_type  | character varying(20)       |           | not null | 

...

Foreign-key constraints:
...

    "fk_ctcataloged_item_type" FOREIGN KEY (cataloged_item_type) REFERENCES ctcataloged_item_type(cataloged_item_type)

Somehow straightening that out would be FABULOUS!

Pretty unrelated, I'm not sure it's entirely intuitive that "Recommended best practice" (from your link) is implemented as "..or we'll nuke EVERYTHING bwahhhahhha......" Maybe that could be rephrased, or the actual expectations and requirements are published somewhere I just haven't found yet, or ???

genevieve-anderegg commented 6 months ago

DMNS Marine Inverts and Trinity College Paleo are the collections that melted Dave's thingee.

Ope. Our one MaterialEntity record is DMNS:Inv:36482, which is a plaster sculpture of a squid (not at all a preserved organism) we sometimes use for education. I think that is within the definition of MaterialEntity. Any way I can help?

dbloom commented 6 months ago

@genevieve-anderegg I'm sure that is the correct use of the term, I just needed some confirmation that things were not misspelled, etc. before I contact GBIF (and in all likelihood, make a fool of myself).

dbloom commented 6 months ago

Posted to GBIF https://github.com/gbif/ipt/issues/2408

AJLinn commented 6 months ago

I'm not sure what any of this means, but I can confirm that the 38511 references to the UAM:EH are correct. :-)

dbloom commented 6 months ago

Ok folks. This has been resolved. I have updated some of the vocabs and extensions in the IPT and everything has published successfully. I assumed, mistakenly, that when we moved to a new infrastructure and updated to the current version of the IPT that my other packages would also update. T'was not so. All good now and everything using MaterialEntity in Arctos is all good.

@dustymc If you are satisfied, you can close this issue.

dustymc commented 6 months ago

Thanks!!