Check and flag legislation that was written by a third party

[waldoj]

The PDF of legislation that was written by a third party is flagged as such. For instance, this bill contains this text at the top:

LEGISLATION NOT PREPARED BY DLS

The process is easy:

• [x] Modify the database to add a column to track whether the legislation was prepared by DLS. Allow that column to be null.
• [ ] Write a PHP script to randomly select X bills for which we have no data for the origin of the text.
• [x] Pipe the already-archived PDF through pdftohtml (part of Poppler, installable with sudo apt-get install poppler-utils).
• [x] Convert the output HTML's entities to ASCII characters with Recode (recode html..utf8)
• [x] Look for the string LEGISLATION NOT PREPARED BY DLS (keeping in mind that those spaces are no-break spaces—that is, use regex's \s, not a literal space.)
• [ ] Update the record in the database to indicate whether the legislation originated externally or not.
• [ ] Include this data within the API.
• [ ] Include this data within the UI, saying nothing if the bill was written internally, but providing a notice if it was written externally.

[waldoj]

This works fine: pdftohtml -f 1 -stdout bill.pdf | recode html..utf8

[waldoj]

I set up external_legislation.sh within /utilities/, to iterate through PDFs in a given directory and return a list of those that contain the text PREPARED.

[waldoj]

Got it. Works great. So far this is just a batch utility, and it works fine. Now to turn this into a little service.