Generate KBART records

[vincentwj]

Recommendation 49. COPIM must review the resources made available on KBART and seek to seek endorsement of its files.

Recommendation 50. Once KBART files are available, COPIM should test with partner institutions that it is available as a package to ‘subscribe’ to in library discovery systems.

[rhigman]

Points of note arising from initial (as yet unreviewed) work under #277 :

• The KBART "standard" is not fully mature, and various sets of guidelines can be found, which are not always clear (e.g. regarding which fields are mandatory) or consistent. The NISO Phase II guidelines appear to be commonly used: https://groups.niso.org/apps/group_public/download.php/16900/RP-9-2014_KBART.pdf
• The Thoth KBART output in its current form was based on an example manually generated by OBP, then improved following feedback from OBP, Jisc and OCLC. Again, the feedback was not always consistent. The KBART Standing Committee was also contacted for feedback, but have not yet had the time to respond; any suggestions they make can be incorporated later.
• Jisc requested that only one author/editor be listed in the first_author/first_editor fields, but OCLC readily accepted an earlier version where all main authors/editors were listed separated by semicolons.
• Jisc suggested that the parent_publication_title_id for a work in a series should either be the series' DOI, or, if unavailable, an internal ID. OCLC suggested using the series' eISSN, as this is commonly done by other providers. (OBP's manually generated KBART used a sequential ID which is not recorded in Thoth; OCLC said it should not be a problem to switch to using eISSNs.)
• Although publication dates are available in Thoth at a resolution of YYYY-MM-DD, Jisc readily accepted the version giving only YYYY, and OCLC specified that YYYY was preferred by libraries.
• OBP requested that the output be sorted by publication date, but the NISO guidelines state it should be sorted alphabetically by title.
• Although Thoth supports non-monograph Work Types such as Book Chapter and Journal Issue, OBP stated that they output all records with the publication_type Monograph, partly due to the very small number of chapters/journals they publish. This logic has been carried over into the KBART output, but may need to be adapted to better fit other publishers' workflows.
• The NISO guidelines state that the filename should be of the format [ProviderName]_[Region/Consortium]_[PackageName]_[YYYY-MM-DD].txt, but this does not match the standard Thoth Export API naming convention, so files will need to be renamed before distribution.
• The NISO guidelines state that date_monograph_published_online and (if applicable) date_monograph_published_print are mandatory fields, and the OCLC guidelines (https://help.oclc.org/Metadata_Services/WorldShare_Collection_Manager/Knowledge_base_collections/Maintain_your_knowledge_base_collections/KBART_values_and_formatting?sl=en) state that title_url is mandatory, therefore the Thoth KBART output (silently) omits any records which are missing a Landing Page or a Publication Date.

[vincentwj]

I guess it makes sense to wait on the NISO standing committee to validate the KBART output before we publicize it? Ideally we can immediately say it's NISO compliant.

[rupertgatti]

Presumably the last point is the sort of thing the 'traffic lights' will flag.

[vincentwj]

I think there is difference between our KBART format compying to NISO standards and the records of a publisher in that format being compliant with it?

[rhigman]

The KBART (NISO) Standing Committee have now also sent feedback, some of which I was able to implement/had already implemented based on other feedback:

• They echoed Jisc in requesting that only one author/editor be listed in first_author/first_editor, and reiterated that date_monograph_published_online and date_monograph_published_print are mandatory (already fixed since sending the example)
• They requested that data cells not be enclosed in quotation marks (fixed under https://github.com/thoth-pub/thoth/pull/277/commits/a4d92242b1b609a668c41456e55df633e82d2a08)
• They pointed out that the filename does not match the NISO guidelines, as above - we will need to resolve this if we eventually set up automated distribution

Some of their feedback was specific to the example data I had sent, which was user-submitted and not always fully accurate/appropriate. I therefore explained the relation between Thoth and individual publishers, and asked whether this would affect the way the endorsement process worked (cf Vincent's comment above).

• Notably, they requested we don't use DOIs for title_id, but instead use the product identifier numbers from title URLs (e.g. 137 for https://www.openbookpublishers.com/product/137), as the example file I had sent was based on OBP data. I explained that not all Thoth publishers would have the same title URL structure, and asked whether DOIs would be acceptable on this basis.

[rhigman]

Closing this card as the initial KBART output generation work is complete. Set up #284 to track the next steps of getting the output endorsed and available for subscription.