As an admin, I want items updated with matching OCLC work URI, best match edition URI, genre, and subjects so that I can include information from OCLC so users will know more about the books.

[rlskoeser]

Notes for testing

• the following cases should be excluded from the reconciliation and should not have OCLC URIs added
  • [x] Items with the following labels in notes: GENERIC, PROBLEM, OBSCURE, ZERO
  • [x] Items with titles ending in *
  • [x] Items that have an existing work URI (check items with existing URIs added by the data team)
• [x] Items with PERIODICAL in notes should be matched with items of type periodical (OCLC search is restricted to material type periodical)
• [x] Items that do not have PERIODICAL should be matched with items of type book (OCLC search specifies material type book)
• Items that are updated with OCLC matches should have the following:
  • [x] work URI
  • [x] edition URI (for best match found)
  • [x] genre
  • [x] subjects (should not include experimental WorldCat subject URIs)
  • [x] format
  • [x] a note in the item history indicating when it was updated by script and what OCLC id was used.

    additional testing May 21

• [x] search is now filtering on language = english and material type not "internet resource"; hopefully this improves genre/subject results
• [x] items where no match is found are flagged in the notes with 'OCLCNoMatch' and a note in the item history indicates when the search was done; it shouldn't have a reconciliation attempted again on subsequent runs of the script until OCLCNoMatch is removed (we may need to test this together)
• [x] genre is now multiple and works like subject, also editable in its own list like subject
• [x] genre and subject are both now editable on the item admin form
• [x] genre list should be included in item CSV export

dev notes

• [x] update OCLC manage command to have two modes, report and update
• [x] revise script to skip items with ZERO in the notes
• [x] script should not update or reconcile items that already have a work URI
• [x] use PERIODICAL in notes to filter on format in OCLC search
• [x] update records when a match is found: work uri, edition uri, genre, subjects, format
• [x] for work subjects: include LLC subjects, FAST headings, and WorldCat person IDs; do not include experimental WorldCat work headings.
• [x] script should create a log entry to document the change when it adds oclc information to an item
• [x] error handling & logging for worldcat entity graph not loading
• [x] add an indicator to notes for items with no match, and document in record history: OCLCNoMatch
• [x] skip items previously marked as no oclc match
• [x] restrict to english language items
• [x] exclude electronic books
• [x] handle multiple genres (like subjects)
• [x] make genre & subject editable on item

[rlskoeser]

Re-estimating to a 5; I think we didn't account for the complexity of handling the subjects. I decided to implement as a many-to-many with a new model, but even without that would have had to resolve the subject URIs to get a useful human-readable label for them.

[clmahoney]

• no reconciliation for GENERIC, PROBLEM, OBSCURE, ZERO; Items with titles ending in *

• I see Items that are updated with OCLC matched with the following: work URI, edition URI (for best match found), format, genre, and subject list.

• I see note in item history that says it was updated by script and the OCLC id that was used

• Items that are not PERIODICAL are matched with book

• PERIODICALS are matched with types of periodicals

• Finding some interesting genres, though: sometimes the genre is in another language besides English

• finding weird subject fields too: "Personal copy" ?

[rlskoeser]

Based on the additional refinements, upgrading from 5pts to 8

[rlskoeser]

@clmahoney I've updated the script based on feedback from you and @jkotin and have re-run it on the test site (I copied over a fresh set of production data and then ran the new version of the script). It's now requesting english language and not electronic editions when it does the OCLC search, but I haven't looked closely to see how much of a difference that's making.

@clmahoney I updated the testing notes to try to indicate what I think you've already signed off on (from your last comment) and added the new things I think you should check.

Here's the summary output from the first time I ran the script:

Processed 7040 items, updated 5331, no matches for 1698

I ran it again to help you test that it's no longer re-checking things that have been previously searched and flagged as no match, here's the output:

Processed 15 items, updated 15, no matches for 0

There were a handful of OCLC data loading errors on the first run that I'm not currently reporting; those were the 15 that it processed on the second one.)

[rlskoeser]

It's not strictly part of this story, so I didn't add it to the testing checklist, but feel free to try: you can go to the list of genres or the list of subjects and remove one that you don't want - removing it will remove it from any books it was associated with, without having to edit all those books individually.

[clmahoney]

• Still seeing non-English language results. But I'm not seeing "electronic books" at all; although I did see an Ebook genre - but I could remove it.
• I see the 'OCLCNoMatch' and a note in the item history indicates when the search was done
• Genre and subject are both editable
• Genre IS included in CSV export

[rlskoeser]

@clmahoney I'm not sure that it's possible to completely fix the non-english results or completely exclude ebooks. Can you tell if the results are better?