christinklez
commented
6 months ago Collection registry #: 11068
CSU SAC validation reports are missing types; it seems it is because there is a wide array of terms they are using for "type," and we do not have everything included in our type vocab set. There are 8 pages (0-8) of vernacular data; I went through pages 0-5; then skimmed 6-7. I think there are a handful we should consider including, but I think we need to really be okay with losing some of these. OR, we can opt to use the Registry "fill" option with "physical object", which it looks like many of these are.
Here are the ones I think we should consider adding:
<dc:type>Photographic print.</dc:type> --> image
<dc:type>Object sculpture</dc:type> --> physical object
<dc:type>Religious objects</dc:type> --> physical object
<dc:type>Ceremonial objects</dc:type> --> physical object
<dc:type>Postcards</dc:type> --> image
<dc:type>Paintings</dc:type> --> image
<dc:type>Animal paintings</dc:type> --> image
<dc:type>Poems</dc:type> --> text
<dc:type>Letters (signs)</dc:type> --> text
Here's my compiled running list of <dc:type> entries -- the bulk of which seem to be physical objects.
page 4:
<identifier>oai:csus.contentdm.oclc.org:jaac/897</identifier><dc:type>Game pieces</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/899</identifier><dc:type>Watercolors</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/918</identifier><dc:type>Lapel pins</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/919</identifier><dc:type>Brooches</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/920</identifier><dc:type>Shell (animal material)</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/922</identifier><dc:type>Stuffed toys</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/923</identifier><dc:type>Object sculpture</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/924</identifier><dc:type>Necklaces</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/925</identifier><dc:type>Arrowheads</dc:type>page 5:
<identifier>oai:csus.contentdm.oclc.org:jaac/1018</identifier><dc:type>Bowls (vessels)</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1019</identifier><dc:type>Ladles</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1020</identifier><dc:type>Turners (culinary utensils)</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1021</identifier><dc:type>Cooking baskets</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1022</identifier><dc:type>Mortars (grinding tools)</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1023</identifier><dc:type>Pestles</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1024</identifier><dc:type>Inkstones</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1025</identifier><dc:type>Handscrolls</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1026</identifier><dc:type>Chopsticks</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1027</identifier><dc:type>Artists' brushes</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1028</identifier><dc:type>Charms</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1029</identifier><dc:type>Wallets</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1031</identifier><dc:type>Dolls</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1032</identifier><dc:type>Abici (calculators)</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1033</identifier><dc:type>Carvings</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1036</identifier><dc:type>Ashtrays</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1037</identifier><dc:type>Wall hangings</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1039</identifier><dc:type>Vases</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1040</identifier><dc:type>Boxes (containers)</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1041</identifier><dc:type>Postcards</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1042</identifier><dc:type>Paintings</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1045</identifier><dc:type>Animal paintings</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1049</identifier><dc:type>Poems</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1052</identifier><dc:type>Letters (signs)</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1056</identifier><dc:type>Earrings</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1064</identifier><dc:type>Medals</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1065</identifier><dc:type>Boxes (containers)</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1069</identifier><dc:type>Ornaments</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1071</identifier><dc:type>Buttons (information artifacts)</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1074</identifier><dc:type>Toys (recreational artifacts)</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1078</identifier><dc:type>Containers</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1087</identifier><dc:type>Decorations (honor)</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1093</identifier><dc:type>Permits</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1094</identifier><dc:type>Legal notices</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1099</identifier><dc:type>Pedestals</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1104</identifier><dc:type>Tins</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1105</identifier><dc:type>Eggbeaters</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1108</identifier><dc:type>Plates (dishes)</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1110</identifier><dc:type>Traditional medicine</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1112</identifier><dc:type>Religious objects</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1122</identifier><dc:type>Talismans; Dolls</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1133</identifier><dc:type>Balls (object genres)</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1142</identifier><dc:type>Figurines</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1147</identifier><dc:type>Prayer books</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1148</identifier><dc:type>Bookstands</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1152</identifier><dc:type>Trophies (objects)</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1153</identifier><dc:type>Holders</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1154</identifier><dc:type>Pens</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1156</identifier><dc:type>Pendants (jewelry)</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1157</identifier><dc:type>Pins (jewelry)</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1163</identifier><dc:type>Mailboxes</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1167</identifier><dc:type>Paperweights</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1173</identifier><dc:type>Insignias</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1174</identifier><dc:type>Patches (costume accessories)</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1181</identifier><dc:type>Buttons (fasteners)</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1183</identifier><dc:type>Billhooks</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1184</identifier><dc:type>Steamers (cookware)</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1201</identifier><dc:type>Character dolls</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1202</identifier><dc:type>Caskets (personal gear)</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1214</identifier><dc:type>Object sculpture</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1225</identifier><dc:type>Baskets</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1226</identifier><dc:type>Knives (culinary tools)</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1228</identifier><dc:type>Ashtrays</dc:type>
<identifier>oai:csus.contentdm.oclc.org:jaac/1231</identifier><dc:type>Fossils</dc:type>page 6: (there are a ton more on this page)
page 7: (and there are more on this page)
<dc:type>Ceremonial objects</dc:type>
Fixes requested
base contentdm mapper issue
is_shown_by: For collection #26723 only. There is a URL transformation that needs to happen at the base contentdm mapper level. Need to fish<dc:identifier>and transform that URL; see the legacy base contentdm mapper: https://github.com/calisphere-legacy-harvester/dpla-ingestion/blob/cfe3dcb06008c0c6cb9d8207fe28bfaa1a855e4f/lib/mappers/contentdm_oai_dc_mapper.py#L31 For example: Fish for:<dc:identifier>http://csus.contentdm.oclc.org/cdm/ref/collection/teaware/id/230</dc:identifier>& transform to:http://csus.contentdm.oclc.org/utils/getthumbnail/collection/teaware/id/230type vocab additions
Resolved issues
[x] fetched, mapped, and actual (CSU DAMS records) do not line up. The Solr and Rikolti counts do not line up. To make things stranger, neither of these match up with the expected number of records in the CSU DAMS. See logs: https://7a8067cb-3b99-477e-a883-7e311175a9b4.c3.us-west-2.airflow.amazonaws.com/dags/validate_by_mapper_type/grid?dag_run_id=manual__2023-11-20T21%3A47%3A47%2B00%3A00&task_id=map_endpoint_task&tab=logs -- this is resolved via the fix to address empty mapped metadata pages.
[x] language: this is a question #11068 Rikolti is mapping language as "['jpn, fre, eng']" -- vernacular hasjpn, fre, eng</dc:language> (instead of three distinct entries) -- okay to accept? but there may be a related enrichment issue with languages. -- this is a question we should ask the contributor.
