Fix the problems with the 477 data

PublicI / broadband-rollout

0 stars 0 forks source link

Fix the problems with the 477 data #3

Closed JoeYerardi closed 6 years ago

JoeYerardi commented 6 years ago

[x] Remove lines containing Python code
[x] Fix encoding problem with accented words
[x] Fix problem with new line characters

JoeYerardi commented 6 years ago

http://www.folkstalk.com/2013/03/sed-remove-lines-file-unix-examples.html

$ grep -n "import os" December-2012-Census_Block.txt
3048020:import os, sys
6373392:import os, sys

$ grep -n "import os" December-2012-Random_pt.txt
1252932:import os, sys
2561916:import os, sys

JoeYerardi commented 6 years ago

The embedded Python code issue has been resolved.

Next up:

ERROR: invalid byte sequence for encoding "UTF8": 0xe9 0x72 0x69 CONTEXT: COPY deployment_2016, line 30930994 ERROR: invalid byte sequence for encoding "UTF8": 0xe9 0x72 0x69 CONTEXT: COPY deployment_2014, line 22292622 COPY 13226419 ERROR: extra data after last expected column CONTEXT: COPY deployment_2012, line 6650658: "7-1-10-1-52197|0000372047|New Edge Network, Inc.|New Edge Networks|130899|New Edge Holding Company|D..."

JoeYerardi commented 6 years ago

Victory is ours, my friends!