UnicodeEncodeError

[GoogleCodeExporter]

What steps will reproduce the problem?
was running from backup2l.sh, died after a few documents:

[...]
spreadsheet:plxrPSqz2Dy-_dVKAY -d->
gdatacopier/gdatacopier-2.0.1/files//sorozatok.ods - UNCHANGED
Traceback (most recent call last):
 File "/home/eszpee/gdatacopier/gdatacopier-2.0.1/gcp.py", line 422, in
<module>
   main()
 File "/home/eszpee/gdatacopier/gdatacopier-2.0.1/gcp.py", line 418, in main
   parse_user_input()                  # Check to see we have the right
options or exit
 File "/home/eszpee/gdatacopier/gdatacopier-2.0.1/gcp.py", line 400, in
parse_user_input
   export_documents(document_source, document_target, options)
 File "/home/eszpee/gdatacopier/gdatacopier-2.0.1/gcp.py", line 202, in
export_documents
   export_filename = target_path + "/" +
sanatize_filename(entry.title.text) + "." + export_extension
 File "/home/eszpee/gdatacopier/gdatacopier-2.0.1/gcp.py", line 71, in
sanatize_filename
   return filename.encode(sys.getfilesystemencoding())
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in
position 11: ordinal not in range(128)

running parameters were: 
/home/eszpee/gdatacopier/gdatacopier-2.0.1/gcp.py -p ******** -o -u
eszpeebackup@gmail.com:/all/all/*
/home/eszpee/gdatacopier/gdatacopier-2.0.1/files/

strangely it runs fine from the shell. 

What version of the product are you using? On what operating system?

ubuntu linux.

Original issue reported on code.google.com by esz...@gmail.com on 24 Oct 2009 at 8:44

[GoogleCodeExporter]

Can you share your shell script with us?

Original comment by devraj on 24 Oct 2009 at 10:18

[GoogleCodeExporter]

If the cron job is being run as a cron job then the shell script may have a 
restricted amount of information 
about the environment such as system encoding resulting in

sys.getfilesystemencoding

not resolving into anything.

Original comment by devraj on 24 Oct 2009 at 10:54

[GoogleCodeExporter]

Could be, but I have no idea how to correct that. :)

Strange thing is that for the first few files it was OK. Altough I have accented
characters in some of the file names, so this could be a reason.

My backup2l.conf script is quite long, the relevant parts are:

    echo "starting GDocs backup"
    sudo -u eszpee /home/eszpee/gdatacopier/gdatacopier-2.0.1/startbackup.sh

and the startbackup.sh script only contains this:

#!/bin/bash
/home/eszpee/gdatacopier/gdatacopier-2.0.1/gcp.py -p ******** -o -u
eszpeebackup@gmail.com:/all/all/* 
/home/eszpee/gdatacopier/gdatacopier-2.0.1/files/

is there anything else I can provide?

Original comment by esz...@gmail.com on 24 Oct 2009 at 11:38

[GoogleCodeExporter]

You were right devraj, it was indeed the problem with the shell environment. 

The following line in my backup script solved everything:

 export LANG=en_US.UTF-8; 

Maybe you could fall back to UTF-8 if you don't find any filesystem encoding, 
but
that's just a nice touch, this is clearly not the fault of the script. Thanks 
for the
hint.

Original comment by esz...@gmail.com on 28 Oct 2009 at 2:45

[GoogleCodeExporter]

Thanks for reporting that in. I might actually add an error message if there 
ins't a LANG variable set, defaulting 
UTF-8 might make it crash for people who require non-UTF8 support.

Original comment by devraj on 28 Oct 2009 at 10:42

• Changed state: Started

[GoogleCodeExporter]

Included check for encoding in environment variables.

Original comment by devraj on 29 Oct 2009 at 8:32

• Changed state: Fixed

[GoogleCodeExporter]

Great, let me know if I can help testing.

Original comment by esz...@gmail.com on 29 Oct 2009 at 8:38