Work on #408 - add Filesystem fetcher

[mjordan]

Github issue: #408

What does this Pull Request do?

Adds a fetcher that uses a directory scan instead of an input file, in effect allowing you to use MIK to generate packages without any metadata. This fetcher is a drop-in replacement for the Csv fetcher in Csv toolchains.

What's new?

• Added src/fetchers/Filesystem.php class file
• Updated some additional classes to avoid the hard-coded reference to the CSV fetcher.
• Added PHP unit tests for this new fetcher, plus an input directory to use in the tests (tests/assets/filesystemfetcher)
• Modified some other test files to explicitly use the Csv fetcher class.

How should this be tested?

This change is testable. To test:

1. Check out the issue-408 branch.
2. Run the PHPUnit tests: phpunit --exclude-group inputvalidators --bootstrap vendor/autoload.php tests. You should get 52 tests, 72 assertions.

Additional Notes

Draft cookbook entry is at https://github.com/MarcusBarnes/mik/wiki/Cookbook:-Using-the-Filesystem-fetcher (toolchain docs not updated yet).

Only works with single-file objects, but we can add compound and books and newspaper issues later.

I've tested this new fetcher on Linux and Windows and it works as intended.

Interested parties

@MarcusBarnes

[mjordan]

I added a post-write hook script that allows files of varying content models (e.g., PDF, large image, video) to be processed at the same time. The script will move the payload file plus the corresponding MODS XML file to a content-model specific directory where they can be ingested via Drush (for example). To use it, add the following setting to your .ini file:

[WRITER]
postwritehooks[] = "/usr/bin/php extras/scripts/postwritehooks/move_packages_by_extension.php"

The output paths in the script will need to be configured to suit local needs.

[MarcusBarnes]

Tested on my environment. PHPunit reports OK (52 tests, 72 assertions) as expected. I still need to test the new post-write hook script.

[mjordan]

Thanks @MarcusBarnes. I'm OK if you skip testing the post-write hook script.

[MarcusBarnes]

@mjordan I tested it and it worked as expected. One question I had was that you actually wanted to move the packages or did you want to copy? I suppose the idea is to have this post-write hook script run last after other post-write scripts. That said, it's a another nice example of post-write functionality in MIK. Thanks!

[mjordan]

My intent was to move the packages, not copy them. Thanks again for reviewing and merging.

[mjordan]

I'll update https://github.com/MarcusBarnes/mik/wiki/Cookbook:-Using-the-Filesystem-fetcher with mention of the move script.

[mjordan]

Done, and linked, along with the Excel fetcher, from the Cookbook home page.