Update the build_data_manifest utility to support single-page image
format documents, for which the pdf2image_regex isn't applicable to
look up cleaned images in S3 from input file.
Issue #, if available: #5
Description of changes:
Update/fix the build_data_manifest utility function to support single-page image format documents.
Previously, this function assumed the pdf2image_regex was applicable to all input documents: But the preprocessing job outputs don't match this format when input raw documents are single images (e.g. PNG, JPEG) or multi-page images (e.g. TIFF). This resulted in errors failing to build collated textract-ref + source-ref manifests for custom corpora including non-PDF documents.
Testing done:
Initially tested with synthetic corpus - exploring additional tests to verify the fix
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Update the build_data_manifest utility to support single-page image format documents, for which the pdf2image_regex isn't applicable to look up cleaned images in S3 from input file.
Issue #, if available: #5
Description of changes:
Update/fix the
build_data_manifestutility function to support single-page image format documents.Previously, this function assumed the
pdf2image_regexwas applicable to all input documents: But the preprocessing job outputs don't match this format when input raw documents are single images (e.g. PNG, JPEG) or multi-page images (e.g. TIFF). This resulted in errors failing to build collatedtextract-ref+source-refmanifests for custom corpora including non-PDF documents.Testing done:
Initially tested with synthetic corpus - exploring additional tests to verify the fix
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.