Add PDF extraction as cloud.gov task(s)

cantsin commented 4 years ago

User story

As an auditor, I want to see automatically extracted text from PDFs before this audit year (2019) so that input to FAC is easier.

Acceptance criteria

[x] We populate any PDF findings (not necessarily pre-2019) in the database
[x] If applicable, run task after midnight EST

Outstanding questions

Memory issues, may need to up the memory/disk quota
Processing may be much slower on cloud.gov
~Compare findings, side-by-side comparison~
- Some findings are not in FAC db, but in the PDF

Implementation notes

Database migration -- separate table, pdf_extract
PDFs manually uploaded to S3 for now

cantsin commented 4 years ago

Update: will not do side-by-side comparison but instead will have two different HTML tables for now. We can reconsider combining the findings with PDF extracts later on when the code is more stable. CC @bpdesigns

cantsin commented 4 years ago

We have PDF extract as a cloud.gov task now, but there are possible improvements we would like to make. Will detail in a separate issue.

bpdesigns commented 4 years ago

Thanks @cantsin

18F / FAC-Distiller