kobotoolbox / kobocat

Our (backend) server for providing blank forms to Collect and Enketo and for receiving and storing submissions.
BSD 2-Clause "Simplified" License
117 stars 126 forks source link

csv_import endpoint does not support Unicode #289

Open jnm opened 8 years ago

jnm commented 8 years ago
  1. Create a project with the attached a.xls form (inside csv_import.zip)
  2. Attempt to import sub.csv using `curl -X POST -u username:password https://kc.kobotoolbox.org/api/v1/forms/[form id]/csv_import -F csv_file=@sub.csv
  3. Notice the resulting 500 error; also notice that the first row of sub.csv (plain) has been imported while the second row (✓✓) has not.

csv_import.zip

Exception info: http://sentry.kbtdev.org/kobo/kobocat/issues/1642/

awemulya commented 5 years ago

@jnm do this error related to. UnicodeDecodeError: 'ascii' codec can't decode byte 0xcd in position 245: ordinal not in range(128) form the following code.

TODO: Something less horrible. This masks storage backend

# `ImportError`s and who knows what else

# ODK validation errors are vanilla errors and it masks a lot of regular
# errors if we try to catch it so let's catch it, BUT reraise it
# if we don't see typical ODK validation error messages in it.
if u"ODK Validate Errors" not in e.message:
    raise

# error in the XLS file; show an error to the user
return {
    'type': 'alert-error',
    'text': unicode(e)
}

https://github.com/kobotoolbox/kobocat/blob/master/onadata/libs/utils/logger_tools.py