search

PalmBeachPost / postgeo

Geocode CSVs and jitter overlapping points
MIT License
23 stars 3 forks source link

cellspacer.py -- shift from binary files (rb/wb to r/w) creates blank lines in Python 2 output #7

Open stucka opened 8 years ago

stucka commented 8 years ago

I'm assuming this works properly in Python 3 else @amikiri wouldn't have submitted patch, but I haven't tested.

michealbeatty commented 8 years ago

Odd, I'm not seeing this issue in either Python 2 or 3. Can I see the source file or a portion thereof?

stucka commented 8 years ago

Should be here: https://gist.github.com/stucka/5e82393945fa7b6cdba7

stucka commented 8 years ago

I'm looking in a text editor, and I've got: data + cr nothing + cr + lf data + cr nothing + cr + lf

michealbeatty commented 8 years ago

OK, I see it happening when I run the script on Windows. Let me look into fixing this.

stucka commented 8 years ago

Looks like it's a thing written circumspectly about in the docs: https://docs.python.org/2/library/csv.html#csv.writer via http://stackoverflow.com/questions/1170214/pythons-csv-writer-produces-wrong-line-terminator

Patched with https://github.com/PalmBeachPost/postgeo/commit/65adc58c6b92fbc5bc79d775e361a4d242679f22 ... and life goes on. =)

stucka commented 8 years ago

Fixed by @amikiri patch in https://github.com/PalmBeachPost/postgeo/commit/c33dcf33b628fa4837264458838e444eb817623a

stucka commented 8 years ago

We're still having problems with this, sadly, such that files created by postgeo.py in Windows aren't readable by cellspacer.py.

I'm thinking the answer might be: Open CSVs with rU Write CSVs with w or possibly wb Don't force csvwriter into any particular behavior

I haven't tested this, however, and my efforts to find best practices have fallen flat.

michealbeatty commented 8 years ago

I'm guessing that adding the same lineterminator argument to csv.writer in postgeo would fix the issue. Let me try that and see if it works.

stucka commented 8 years ago

@amikiri , what happens if you just try with rU and wb?

I tried with rU and w (no b) and got cr-cr-lf with no line terminator argument.

stucka commented 8 years ago

I pushed out a bunch of smallish fixes to postgeo.py in commit https://github.com/PalmBeachPost/postgeo/commit/f3d911144e110df1e32665b55e16ab0d4d121784 , but also tried the rU, wb, no terminator. Works here. No idea if it works on your end. I can test in Linux but don't have easy access to a Mac box.

michealbeatty commented 8 years ago

The 'wb' causes an error in Python 3 because it's expecting a real bytes object not a string. However leaving off the 'b' worked fine (on my Mac). Did the lineterminator fix not work on your Windows machine?