search

eXtensibleCatalog / OAI-Toolkit

Synchronize MARC metadata in most ILSs with XC
5 stars 1 forks source link

Voyager extract scripts, newest to be used #148

Open patrickzurek opened 8 years ago

patrickzurek commented 8 years ago

JIRA issue created by: rcook Originally opened: 2012-07-23 11:54 AM

Issue body: Just want to make sure that 1. Ralph's extract scripts (daily, initial, deletes, etc.) are documented and 2. that the CARLI staff knew about them and were using them. Specifically a question came up today about whether a data set used in testing contained suppressed records (as a possible reason why a single institution database contained so many match points in Aggregation within itself).

I know Ralph has quite an involved process to handle suppressed records and thought it worth having Gordon touch base.

[~cdelis] [~admin] [~jbrand]

patrickzurek commented 8 years ago

JIRA Coment by user: fellows JIRA Timestamp: 2012-08-01 02:47 PM

Comment body:

No, I am not using any XC supplied scripts to export records from voyager for the OAI toolkit. I am using a script I wrote myself. The only scripts I can find on the WebSite (http://code.google.com/p/xcoaitoolkit/downloads/list) are from May 2010. Are these the ones you are referring to?

The script I wrote exports the bib and MFHDs separately and does two marcexports for bibs (one for updates/new, one for suppressed bibs). It turns the suppressed bibs to deletes, combines them with the true deletes, and sends a delete file to OAI.

patrickzurek commented 8 years ago

JIRA Coment by user: rcook JIRA Timestamp: 2012-08-01 02:57 PM

Comment body:

[~admin] can you direct/point us on this? Are all doing the same basic thing? Are there separate needs that require two processes? If so, great, but otherwise maybe we can consolidate better and be more transparent.

patrickzurek commented 8 years ago

JIRA Coment by user: admin JIRA Timestamp: 2012-08-01 03:24 PM

Comment body:

I'm in the process of updating my documentation for the export process as the scripts in the OAI wiki are out of date. We're doing something very similar- exporting four different files daily (updates/new & suppressed for both bib and MFHDs) then a weekly deleted export that contains separate bib & MFHDs files. We could actually pull out the deletes daily as well if we wanted to.

patrickzurek commented 8 years ago

JIRA Coment by user: fereira JIRA Timestamp: 2012-11-16 07:23 AM

Comment body:

I've been working on something for extracting bib, mfhd, and authority records from a voyager database that can be run more frequently than a daily update. I'm having a heck of time trying to get something working that will read the CLOB in the database such that it produces marc that the marc4j library reader can handle. If anyone has any code that is accessing the MARC data from the voyager database directly I'd love to take a look at it.