Issue #441 - Change metadata manipulators in OAI toolchains to manipulate entire XML document

[mjordan]

Github issue: (#441)

What does this Pull Request do?

Modifies metadata parsers used in OAI-PMH toolchains to use metadata manipulators that have access to the entire metadata XML file.

What's new?

• Modifies the DC and MODS OAI-PMH metadata parsers to use metadata manipulators that have access to the entire serialized XML metadata document instead of Cdm and Csv-style XML snippets.
• Provides a new metadata manipulator, SimpleReplaceOai.

How should this be tested?

The changes in this PR do not have PHPUnit tests, they must be smoketested.

• Check out the issue-441 branch.
• Run composer dump-autoload or the eqivalent on your system.
• Using the attached .ini file, run mik: ./mik -c issue-441.ini
  • The DC files in the output directory should say "Super Special" in their dc.rights elements.
• Edit the .ini file so that you are harvesting MODS instead of DC (uncomment the metadata_prefix = mods and class = mods\OaiToMods lines).
• Delete your output and temp directories and rerun MIK.
  • The MODS files in the output directory should say "Super Special" in their <accessCondition> elements.

Additional Notes

• A wiki entry documenting the new metadata maniipulator is available at https://github.com/MarcusBarnes/mik/wiki/Metadata-manipulator:-SimpleReplaceOai. It should be linked from https://github.com/MarcusBarnes/mik/wiki/Manipulators and all of the OAI-PMH toolchain wiki pages.

Interested parties

@MarcusBarnes @bondjimbond

issue-441.ini.txt

[bondjimbond]

Tried the first run and got Problem Records for every record. XML files were acquired, but no OBJ files downloaded.

Sample error:

[2017-10-27 13:57:23] ErrorException.ERROR: ErrorException {"message":"problem writing package","record_key":"oai%3Adigital.lib.sfu.ca%3Ahiv_1","details":"[object] (GuzzleHttp\Exception\RequestException(code: 0): No system CA bundle could be found in any of the the common system locations.\nPHP versions earlier than 5.6 are not properly configured to use the system's\nCA bundle by default. In order to verify peer certificates, you will need to\nsupply the path on disk to a certificate bundle to the 'verify' request\noption: http://docs.guzzlephp.org/en/latest/clients.html#verify. If you do not\nneed a specific certificate bundle, then Mozilla provides a commonly used CA\nbundle which can be downloaded here (provided by the maintainer of cURL):\nhttps://raw.githubusercontent.com/bagder/ca-bundle/master/ca-bundle.crt. Once\nyou have a CA bundle available on disk, you can set the 'openssl.cafile' PHP\nini setting to point to the path to the file, allowing you to omit the 'verify'\nrequest option. See http://curl.haxx.se/docs/sslcerts.html for more\ninformation. at /Users/Brandon/mik/vendor/guzzlehttp/guzzle/src/Exception/RequestException.php:52, RuntimeException(code: 0): No system CA bundle could be found in any of the the common system locations.\nPHP versions earlier than 5.6 are not properly configured to use the system's\nCA bundle by default. In order to verify peer certificates, you will need to\nsupply the path on disk to a certificate bundle to the 'verify' request\noption: http://docs.guzzlephp.org/en/latest/clients.html#verify. If you do not\nneed a specific certificate bundle, then Mozilla provides a commonly used CA\nbundle which can be downloaded here (provided by the maintainer of cURL):\nhttps://raw.githubusercontent.com/bagder/ca-bundle/master/ca-bundle.crt. Once\nyou have a CA bundle available on disk, you can set the 'openssl.cafile' PHP\nini setting to point to the path to the file, allowing you to omit the 'verify'\nrequest option. See http://curl.haxx.se/docs/sslcerts.html for more\ninformation. at /Users/Brandon/mik/vendor/guzzlehttp/guzzle/src/functions.php:197)"} []

[bondjimbond]

Though apparently the same error cropped up on the Master branch (after disabling the new manipulator). So I guess I can't test this until the error is fixed.

[bondjimbond]

I'll wait a bit to see if @MarcusBarnes wants to review; if not, I'll merge.

[bondjimbond]

@mjordan By the way, what would you think of adding the Reminder Bot to MIK?

[mjordan]

Sounds good. @MarcusBarnes did you see the options for that?

    /remind [me OR @othersuer] [reminder text. "to" is optional] in [time in minutes, hours, days, etc.]

Example:

    /remind me to merge in 24 hours

Result 24 hours later is notification from Reminders Bot: 

     \uD83D\uDC4B @bondjimbond, merge

Example: 

    /remind @dhlamb to look at this in 30 minutes

Result 30 minutes later from Reminders Bot:

     \uD83D\uDC4B @dhlamb, look at this 

[mjordan]

@MarcusBarnes mind if @bondjimbond merges this?

@bondjimbond let's move the Reminder Bot to a new issue. I can do that if you want, or you go for it.

[MarcusBarnes]

I trust @bondjimbond to merge as appropriate.

Please move the Reminder Bot to a new issue.