Open dkirkby opened 9 years ago
If we decide to go with something like CSV, the same change should probably be made to the default output from bossdata ... --save-data ...
.
There doesn't seem to be an option I can find to simply force quoting of all text data. A hacky solution with some drawbacks:
On output, insert
for i in table.columns:
if str(table[i].dtype)[0:2] == '|S':
table[i].format = '\'%s\''
before the current
table.write(args.save, format='ascii')
All strings get single quoted. But then reading in has to be changed to:
table = astropy.table.Table.read(..., format='ascii', quotechar='\'')
This last bit is annoying, and working around the API rather than through it...
We ideally want the default format to be good compromise between human- and machine-readable. If we only had to deal with numeric values, the existing format would be ok. Perhaps a standard separator (i.e., CSV) is a better compromise than quoting all strings if that requires special reader args or unquoting logic (especially if the string has embedded quotes...) I wonder how CSV handles strings with embedded commas?
The following query produces output that is difficult to read back in:
The resulting file is (assuming a
.txt
or.dat
extension) written byastropy.table
using:and contains:
Unfortunately,
Table.read(..., format='ascii')
is unable to read this back because it reasonably deduces that the second column is an integer when it is actually a string.