UTF-8 support for .csv file

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Add non-english (non-ascii) symbols into lecture name in .csv file
2. Try to load this lecture in browser

What is the expected output? What do you see instead?
Exception is throws by python

Original issue reported on code.google.com by hyp...@gmail.com on 22 Jan 2013 at 12:00

[GoogleCodeExporter]

I confirm this is a real issue. It looks like Python CSV reader does not like 
UTF-8, which is a surprise 
(http://docs.python.org/2/library/csv.html#csv-examples).

Original comment by psimakov@google.com on 24 Jan 2013 at 10:20

[GoogleCodeExporter]

I solved this problem by converting non-ascii symbols into html/xml tags.
In tools/verify.py the first loop of read_objects_from_csv method I added one 
line:

row = [repr(cell.decode("utf-8").encode('ascii', 
errors='xmlcharrefreplace'))[1:-1] if isinstance(cell, basestring) else cell 
for cell in row]

line 6 here: http://pastebin.com/26pJysna

It solves the problem for me, but not sure if this correct way.

Original comment by hyp...@gmail.com on 24 Jan 2013 at 10:23

[GoogleCodeExporter]

Yes, it works, but I have a shorter version:

row = [cell.decode("utf-8").**encode('ascii',
errors='xmlcharrefreplace') if isinstance(cell, basestring) else cell for
cell in row]

Original comment by psimakov@google.com on 25 Jan 2013 at 8:28

[GoogleCodeExporter]

Fixed in 1.3.0

Original comment by psimakov@google.com on 19 Mar 2013 at 11:15

• Changed state: Fixed