Mac vs. PC CSV files

[redbluewater]

This might just belong in the wiki, but I just discovered that domdb does not like CSV files that are created by a mac. I was able to fix this be re-saving the CSV and selecting 'MS-DOS'.

[joefutrelle]

Will check to see if this is a "feature" of csv.DictReader in which case I should probably look into using something more robust such as pandas.read_csv.

[joefutrelle]

K, so were you using Mac Excel to save the CSV? And used the option "Windows Comma-Separated"? And that didn't work so then you used "MS-DOS Comma Separated"? Or did you create the CSV using some tool other than Excel?

[joefutrelle]

... I'm seeing opposite-than-reported behavior in that when I save from Mac Excel as "Windows Comma Separated" csv.DictReader works, but in "MS-DOS Comma Separated" it fails with this helpful error message:

vagrant@vagrant-ubuntu-trusty-64:/vagrant/scratch$ python read_csv.py 
Traceback (most recent call last):
  File "read_csv.py", line 4, in <module>
    for d in DictReader(fin):
  File "/usr/lib/python2.7/csv.py", line 107, in next
    self.fieldnames
  File "/usr/lib/python2.7/csv.py", line 90, in fieldnames
    self._fieldnames = self.reader.next()
_csv.Error: new-line character seen in unquoted field - do you need to open the file in universal-newline mode?

Googling "universal newline mode Python" gives me this answer:

http://stackoverflow.com/questions/6726953/open-the-file-in-universal-newline-mode-using-csv-module-django

Should be easy to apply that fix

[redbluewater]

Someone else in the lab generated the problem file on a mac. On the mac, she was using MATLAB to add information to the CSV files. When I saw the error message in domdb, I suspected this was a mac/pc/file issue which is why I opted to open the files in Excel and re-save them as I described above. Glad to hear this will be an easy fix.

[joefutrelle]

Got it, I can attempt to test this now using Mac MATLAB--should be no problem now.

[redbluewater]

I just checked the latest version of the code with Mac-created file and we seem to be all set for now.