Create Master QA script with Optional Config File

jarmoza commented 1 year ago

The idea here is to build a master script for QAing our cropping methods, but also something that can be extended for other parts of the book ingestion pipeline like line extraction.

[x] The script should be able to take arguments for limited runs (i.e. one book directory) as well as read arguments from a config file for more extensive runs (i.e. a directory of books).
[x] For the autocrop QA, this will replace the master shell script that was being used: autocrop_qa_fullrun.sh.
[x] The script should take mandatory arguments like qa type and book directory, optional arguments like output directory, and even variable arguments like a list of serial commands to run (i.e. clear old output, run qa, collate results)

jarmoza commented 1 year ago

A sample of what a config yaml will contain:

BOOKS_DIRECTORY: "/ocean/projects/hum160002p/shared/books/test_autocrop/"
OUTPUT_DIRECTORY: "/ocean/projects/hum160002p/shared/books/test_autocrop/output/"
QA_TYPE: "autocrop"
COMMANDS: ["clear_output", "run_qa", "collate_results"]

jarmoza commented 1 year ago

Stable and mostly complete (for this iteration) as of https://github.com/printprobability/qa-workflow/commit/3d0e66813934584d504c3f0f36fa444e05dda9a3

printprobability / qa-workflow

Create Master QA script with Optional Config File #3