search

ucd-cws / PISCES

Software for tracking and analyzing fish species range data
Other
0 stars 2 forks source link

CSV writer in tables.py needs to strip newlines to prevent parsing errors on web server side #2

Closed nickrsan closed 12 years ago

nickrsan commented 12 years ago

Original report by Nicholas Santos (Bitbucket: nickrsan, GitHub: nickrsan).

Problem description from amritsub

"One problem is one entry in huc12fullstate was split into two separate lines for some reason. All the other records were imported, it was just that one record that wasn't getting imported since it was broken up between two lines. I can tell you what record it was exactly tomorrow. Another problem was that in the observations table, in the other_data column, there were unescaped new lines in the entries for the column. I either need those newline characters gone or I need the newline characters and all other special characters to be escaped within entries. I will see if species_aux works tomorrow."

nickrsan commented 12 years ago

Original comment by Nicholas Santos (Bitbucket: nickrsan, GitHub: nickrsan).

I should be able to make newlines disappear pretty easily. I'll just do a parse for \n, \r, and \r\n and remove it (maybe replacing it with something like ///) - special characters are trickier - if you come across any particular ones, definitely let me know and I'll see what we can do about removing it. I can also just not include the other_data column, which is probably the only problem child in the set.

nickrsan commented 12 years ago

Original comment by Nicholas Santos (Bitbucket: nickrsan, GitHub: nickrsan).

Resolved and added arguments.