Possible Issue with UTF-8 filenames

[xybu]

When I try to upload a file whose name is UTF-8 encoded, a ProtocolError is raised:

xb@ubuntu:~/OneDrive$ onedrive-cli put 中文.txt
Traceback (most recent call last):
  File "/usr/local/bin/onedrive-cli", line 9, in <module>
    load_entry_point('python-onedrive==14.06.3', 'console_scripts', 'onedrive-cli')()
  File "/usr/local/lib/python2.7/dist-packages/onedrive/cli_tool.py", line 323, in main
    resolve_path(optz.folder), overwrite=not optz.no_overwrite)
  File "/usr/local/lib/python2.7/dist-packages/onedrive/api_v5.py", line 391, in put
    method='post', files=dict(file=(name, src)))
  File "/usr/local/lib/python2.7/dist-packages/onedrive/api_v5.py", line 329, in __call__
    return self.request(api_url(), **kwz)
  File "/usr/local/lib/python2.7/dist-packages/onedrive/api_v5.py", line 176, in request
    raise raise_for.get(code, ProtocolError)(code, message)
onedrive.api_v5.ProtocolError: (400, {u'message': u'The request entity body has an incorrect value in the \'Content-Disposition\' header. The expected format for this value is \'Content-Disposition: form-data; name="file"; filename="[FileName]"\'.', u'code': u'request_body_invalid'})

Could you tell me how to fit such kinds of file names within your API and cli program?

[mk-fg]

Hm, not sure if filenames in the POST body matter, will need to check. And make sure module doesn't mangle these somehow (causing the error), otherwise also for other ways to upload stuff (iirc there was PUT for something similar...).

One workaround with current code might be to upload the file, then rename it, though API might reject names there, of course, if it's a legitimate limitation for filenames there.

[xybu]

It seems the issue resides in urllib3 or request. I try to dive into the issue, starting from http://msdn.microsoft.com/en-us/library/dn659726.aspx and found a similar problem with Box python API https://github.com/sookasa/box.py/issues/30 and finally I found http://stackoverflow.com/questions/20591599/why-arent-post-names-with-unicode-sent-correctly-when-using-multipart-form-data

May we have a workaround in the API layer? Or should we raise it to urllib3 package?

[mk-fg]

I think both reporting it to urllib3/requests devs and having a workaround would be the best, though given age of that stackoverflow issue, I suspect requests might have such bug opened or closed already. There's some monkey-patching for TLS 1.0 thing in older requests versions as it is, don't mind adding anything that works.

Though maybe it'd just be easier to use PUT instead of POST for file uploads, as indicated by http://msdn.microsoft.com/en-us/library/dn659726.aspx - just swap one with the other, check that url encoding works correctly and bingo! Iirc there were some special hacks for POST to set Content-Type (as API wants it) and such, which might apply to PUT as well.

Given a bit of work-related stuff I currently have, suspect I might be too lazy to look into the thing myself in the next few days.

[Tronic69]

In a post at https://github.com/xybu92/onedrive-d/issues/44 you said "Python 2.x does not handle Unicode file names seamlessly as Python 3 does". That means filenames will be treted correct, if I have installed Python 3?

I have inastalled Python 2.7.6 and 3.4.0, but when calling just python then 2.7.6 will be used:

$> python -V Python 2.7.6

$> python3 -V Python 3.4.0

How could I tell onedrive-d to use python3 instead of python? Maybe that fixes my problems.

[mk-fg]

Ugh, I forgot about this issue...

[xybu]

Please think of py2 and py3 as different things. python-onedrive and onedrive-d are written in py2.

Maybe do 2to3 at some time?

Sincerely, Xiangyu Bu

Sent from my Windows Phone

---

From: Tronic69mailto:notifications@github.com Sent: ‎7/‎30/‎2014 2:21 AM To: mk-fg/python-onedrivemailto:python-onedrive@noreply.github.com Cc: Xiangyu Bumailto:xybu92@live.com Subject: Re: [python-onedrive] Possible Issue with UTF-8 filenames (#19)

In a post at https://github.com/xybu92/onedrive-d/issues/44 you said "Python 2.x does not handle Unicode file names seamlessly as Python 3 does". That means filenames will be treted correct, if I have installed Python 3?

I have inastalled Python 2.7.6 and 3.4.0, but when calling just python then 2.7.6 will be used:

$> python -V Python 2.7.6

$> python3 -V Python 3.4.0

How could I tell onedrive-d to use python3 instead of python? Maybe that fixes my problems.

---

Reply to this email directly or view it on GitHub: https://github.com/mk-fg/python-onedrive/issues/19#issuecomment-50577865

[mk-fg]

Welp, github closed it automatically from commit msg.

Can you guys maybe test it as well? Seem to work for me now with filename mentioned in the thread-starter, as filename now gets encoded in the URL:

DEBUG:requests.packages.urllib3.connectionpool:"PUT /v5.0/folder.[...]/files/%E4%B8%AD%E6%96%87.txt?overwrite=ChooseNewName? HTTP/1.1" 201 270

Just needed embarassingly simple one-liner fix, as mentioned in the comment above - using PUT instead of POST. Docs even suggest using that (at least now), probably missed it before.

[mk-fg]

Ok, guess it should be fixed.