Make report_id Windows-safe, and validate

[divergentdave]

I've been cleaning these up as I go along, but we should make sure that all the scrapers are generating report_ids that are valid filenames on Windows, and then add checks to enforce that in validate_report(). In particular, we should check for these characters: \ / : * ? " < > | (and newline, and carriage return) and make sure that the length isn't too long.

[konklone]

Good call - there is a section to do that, but it only checks for /. Updating it for the remaining characters is a good idea.

Alternatively, the report_id field could be pre-processed so that any of the invalid characters are auto-replaced with -. This runs the risk of collisions, though.

[divergentdave]

Here's a list of problem path names I ran into on a recent test run.

• [x] 'data\doj\2006\http:--www.justice.gov-archive-ag-annualreports-pr2005-TableofContents'
• [x] 'data\epa\2002\10960-2002-\r\n M-000031'
• [x] 'data\fdic\2013\Overview-of-FDIC-OIG-Report:-Comprehensive-Study-o'
• [x] 'data\flra\2005\Audit-Report:-FLRA-Security-Programs-(September-2004)---Non-Public-Report'
• [x] 'data\ncua\2012\Report-of-Investigation:-Herbert-S.-Yolles---State-Employees-Credit-Union'
• [x] 'data\tva\2005\Upon Procedures: TVA Fiscal Year 2005 Performance Measures - 2005-063F'
• [x] 'data\usaid\2012\Afghanistan-Pakistan-Oversight-Report:-Jul---Sept-2012'

[konklone]

I see all the checkboxes are checked - is this good to review/merge?

[divergentdave]

I have one more commit to add with the validation end of this

[konklone]

Nice! :+1: