Add Local Lookup for RDA Terms

[sfolsom]

Samvara QA code has a sample yml file to add a controlled list locally to a QA instance: https://github.com/samvera/questioning_authority/blob/main/config/authorities/states.yml

RDA Registry doesn't seem to have a search API that we can use to create a direct lookup, so we need to add a file for each of the RDA Reference value vocabularies found here: https://www.rdaregistry.info/termList/. The ids in the yml will need to be the URI for the term, and the term in the yml should be the preferred term in English.

I'm not sure if our instance is set up for these types of lookups, so we'll have to do some testing.

• [x] RDA Aspect Ratio Designation
• [x] RDA Bibliographic Format
• [x] RDA Broadcast Standard
• [x] RDA Carrier Extent Unit
• [x] RDA Carrier Type
• [x] RDA Cartographic Data Type
• [x] RDA Collection Accrual Method
• [x] RDA Collection Accrual Policy
• [x] RDA Colour Content
• [x] RDA Configuration of Playback Channels
• [x] RDA Content Type
• [x] RDA Extension Plan
• [x] RDA File Type
• [x] RDA Font Size
• [x] RDA Form of Musical Notation
• [x] RDA Form of Notated Movement
• [x] RDA Form of Tactile Notation
• [x] RDA Format of Notated Music
• [x] RDA Frequency
• [x] RDA Generation
• [x] RDA Groove Pitch of an Analog Cylinder
• [x] RDA Groove Width of an Analog Disc
• [x] RDA Illustrative Content
• [x] RDA Interactivity Mode
• [x] RDA Layout
• [x] RDA Linked Data Work
• [x] RDA Material
• [x] RDA Media Type
• [x] RDA Mode of Issuance
• [x] RDA Polarity
• [x] RDA Presentation Format
• [x] RDA Production Method
• [x] RDA Recording Medium
• [x] RDA Recording Methods
• [x] RDA Recording Source
• [x] RDA Reduction Ratio Designation
• [x] RDA Regional Encoding
• [x] RDA Scale Designation
• [x] RDA Sound Content
• [x] RDA Special Playback Characteristics
• [x] RDA Status of Identification
• [x] RDA Terms
• [x] RDA Track Configuration
• [x] RDA Type of Binding
• [x] RDA Type of Recording
• [x] RDA Unit of Time
• [x] RDA User Tasks
• [x] RDA Video Format

[sfolsom]

If I create a yml file for one as a proof of concept that we can test, perhaps we can script the translation of RDF descriptions of these terms to the yml file that QA needs.

[sfolsom]

Confirmed again that there's no API.

[chrisrlc]

@sfolsom You mentioned: "The ids in the yml will need to be the URI for the term, and the term in the yml should be the preferred term in English.", but it looks like we can include separate id and uri fields for each term. For something like: http://www.rdaregistry.info/termList/AspectRatio, I can create a yml with the following:

:terms:
    - :id: "1001"
      :term: full screen
      :uri: http://rdaregistry.info/termList/AspectRatio/1001
    - :id: "1003"
      :term: mixed aspect ratio
      :uri: http://rdaregistry.info/termList/AspectRatio/1003
    - :id: "1002"
      :term: wide screen
      :uri: http://rdaregistry.info/termList/AspectRatio/1002

Would that work? Otherwise, using a uri for id seems to break individual term fetching as is (e.g. /authorities/show/local/rda_aspect_ratio_designation/1001 vs /authorities/show/local/rda_aspect_ratio_designation/http://rdaregistry.info/termList/AspectRatio/1001).

[chrisrlc]

Also, do we want to include all terms in the local lists, including deprecated terms? Example of vocab with deprecated and published terms: https://www.rdaregistry.info/termList/RDATerms/

[sfolsom]

Good questions! I think we shouldn't bother with a fetch test for these (which is really just a test that the API is up and running) since these are "hard coded" into the lookup service with the yml file and not connecting to an external service. If the id: value is the URI is the same as the cities pick list, I think that would be ok.

I think for now we can/should include the deprecated terms so that catalogers who are familiar with a term can see that it's deprecated, and should use something else.

[chrisrlc]

Sounds good - will change it so that id and uri are the same:

:terms:
    - :id: http://rdaregistry.info/termList/AspectRatio/1001
      :uri: http://rdaregistry.info/termList/AspectRatio/1001
      :term: full screen
    - :id: http://rdaregistry.info/termList/AspectRatio/1003
      :uri: http://rdaregistry.info/termList/AspectRatio/1003
      :term: mixed aspect ratio
    - :id: http://rdaregistry.info/termList/AspectRatio/1002
      :uri: http://rdaregistry.info/termList/AspectRatio/1002
      :term: wide screen

Fwiw, there seem to be 3 different endpoints for connecting to local controlled vocabulary:

1. Search: https://lookup.ld4l.org/authorities/search/local/publisher_cities_select_list?q=London
   • Requires query (q) parameter. Only returns id, label and optional uri.
   • Also responds to page_limit and page_offset params
2. All terms: https://lookup.ld4l.org/authorities/terms/local/publisher_cities_select_list
   • Returns all terms with id, label, optional uri, an "active" boolean field (defaults to true unless set to false in yml)
3. Individual term fetch: no example on lookup.ld4l because all the existing local controlled vocabulary ymls use uri as ids, but the endpoint would look something like: http://localhost:3001/authorities/show/local/rda_aspect_ratio_designation/1001
   • Returns all fields for a given term, including any arbitrary fields set in the yml file and an "active" boolean field (defaults to true unless set to false in yml)

So a benefit of the individual term fetching endpoint for local controlled vocabulary is that we could theoretically add additional information for each term that wouldn't show up in search.

We could also set that "active" boolean field to false for deprecated terms - terms with active: false still show up in search.

No changes suggested, just some context I found when digging around that I thought could be useful!

[chrisrlc]

Ready for testing on lookup-int!

There should be 48 new local lookup endpoints, each local subauth in the endpoint urls is the name of the vocab but in lowercase snakecase.

For example. the endpoints for RDA Video format is:

1. Search: https://lookup-int.ld4l.org/authorities/search/local/rda_video_format?q=beta
2. All terms: https://lookup-int.ld4l.org/authorities/terms/local/rda_video_format
3. Individual term fetch: not working because id is a uri.

RDA Aspect Ratio Designation:

1. Search: https://lookup-int.ld4l.org/authorities/search/local/rda_aspect_ratio_designation?q=screen
2. All terms: https://lookup-int.ld4l.org/authorities/terms/local/rda_aspect_ratio_designation
3. Individual term fetch: not working because id is a uri.

Etc. Please let me know if you'd like any changes/if you notice anything strange!

[sfolsom]

I've started taking a look at these, and the search results look good. (I'm still going to go through each of the 48, but one thing I've noticed so far is that there's no "uri": in the json. I think this is what Sinopia is looking for.)

[chrisrlc]

I noticed the missing uri also wasn't showing up for https://lookup.ld4l.org/authorities/search/local/publisher_cities_select_list?q=London either, so wasn't sure if that was just expected behavior. Oddly, uri and id are showing up in the json in my local environment though – I'll poke around to try to figure out what the difference is.

[sfolsom]

I just looked at the cities lookup in Sinopia, and it's not adding the URI to records when I try to link to them. Want me make a separate issue to add "uri": to that lookup?

[chrisrlc]

The upgraded qa gem has been deployed to lookup-int, and I've confirmed that the uri fields are now displaying in the json response, e.g. https://lookup-int.ld4l.org/authorities/search/local/rda_aspect_ratio_designation?q=screen. But if Sinopia is still sad about the updated response, please let me know!