search

influence-usa / lobbying_federal_domestic

Public domain data collectors for federal domestic lobbying disclosures. https://github.com/unitedstates-influence/lobbying-federal-domestic/wiki
Other
11 stars 5 forks source link

determine how to treat bad encoding in original documents #8

Closed boblannon closed 10 years ago

boblannon commented 10 years ago

example: 

xmlParseCharRef: invalid xmlChar value 16, line 277, column 44
/home/blannon/dev/influence-usa/lobbying-federal-domestic/data/original/house_xml/LD2/2013/Q4/300616796.xml

this was a  in the LOBBYINGDISCLOSURE2/alis/ali_info/specific_issues/description field, which was either meant to be a linefeed or a DLE char.

boblannon commented 10 years ago

for sopr_html, addressed with clean_text parser. see e3785693ebe