Open llimeht opened 1 year ago
I have never encountered problems with csv.writer
inserting "lots of blank lines" when opening a file the normal way. I wonder if there is some other reason for these blank lines that is machine or OS specific? If there appears to be a newline every second line, it might be a problem with whatever you are using to open the files interpreting \r\n
as two new lines.
[Edit] I stand corrected, this is the recommended way to use csv.writer, from the docs:
If newline='' is not specified, newlines embedded inside quoted fields will not be interpreted correctly, and on platforms that use \r\n linendings on write an extra \r will be added. It should always be safe to specify newline='', since the csv module does its own (universal) newline handling.
@alchemyst the failure is indeed OS-specific. It's fine on linux (and presumably macos) and the output file is an utter mess on windows.
On Linux:
$ ipython3
Python 3.10.8 (main, Nov 4 2022, 09:21:25) [GCC 12.2.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.5.0 -- An enhanced Interactive Python. Type '?' for help.
In [1]: import csv
In [2]: with open("test.csv", "w") as fh:
...: writer = csv.writer(fh)
...: writer.writerow(['a', 'b'])
...: writer.writerows([[1, 2], [3, 4], [5, 6]])
...:
In [3]: with open("test.csv", "rb") as fh:
...: print(repr(fh.read()))
...:
b'a,b\r\n1,2\r\n3,4\r\n5,6\r\n'
But on windows:
> ipython3
Python 3.10.4 | packaged by conda-forge | (main, Mar 30 2022, 08:38:02) [MSC v.1916 64 bit (AMD64)]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.4.0 -- An enhanced Interactive Python. Type '?' for help.
In [1]: import csv
In [2]: with open("test.csv", "w") as fh:
...: writer = csv.writer(fh)
...: writer.writerow(['a', 'b'])
...: writer.writerows([[1, 2], [3, 4], [5, 6]])
...:
In [3]: with open("test.csv", "rb") as fh:
...: print(repr(fh.read()))
...:
b'a,b\r\r\n1,2\r\r\n3,4\r\r\n5,6\r\r\n'
Changing the above to use using newline=""
produces the same CRLF-terminated output on both platforms.. (I assume it's CRLF-terminated even on Linux because the default for the csv
module is dialect="excel"
).
This patch adds three simple improvements to the
to_csv()
method of the Historian:to_csv()
withnewline=""
to prevent lots of blank lines in the output. See https://docs.python.org/3/library/csv.html#csv.writerto_csv()
to also accept file-handles and file-like objects so that data can be written toStringIO
etcto_csv()