--no-convert option not honored

[GoogleCodeExporter]

What steps will reproduce the problem?

>python google docs upload --folder BACKUP "C:\...\KeePass John Navas.kdb"

   OR

>python google docs upload --folder BACKUP --no-convert "C:\...\KeePass John 
Navas.kdb"

What is the expected output? 

Upload success! Direct link: http://docs.google.com/document/edit?id=...

What do you see instead?

Failed to upload C:\...\KeePass John Navas.kdb
{'status': 400, 'body': 'Could not convert document.', 'reason': 'Bad Request'}

What version of the product are you using? 

0.9.7 + folder patch

On what operating system? 

Windows 7 Home Premium (with all updates)

What version of gdata-python-client (aka python-gdata)?

2.0.10

Please provide any additional information below.

Python 2.6.5

COMMENTS: This worked on June 23, but fails today (June 28), with or without 
"--no-convert".

Original issue reported on code.google.com by john.na...@gmail.com on 29 Jun 2010 at 12:29

[GoogleCodeExporter]

* The same procedure uploads a different file successfully:

>python google docs upload --folder BACKUP "C:\...\clipdat2.rdf"
Loading C:\...\clipdat2.rdf
Upload success! Direct link: http://docs.google.com/document/edit?id=...

* Replacing spaces in the file name with underscores did not help.

Original comment by john.na...@gmail.com on 29 Jun 2010 at 12:40

[GoogleCodeExporter]

UPDATE: The issue appears to be failure to honor the --no-convert option.

I just noticed that the version of this file uploaded on June 23 has a Google 
Document icon, not the plain white page icon of a non-coverted upload through 
the Web interface.  Likewise the other file that still uploads successfully.  I 
then checked the "google" script and found:

parser.add_option('--no-convert', dest='convert',
                    action='store_false', default=True,
                    help='Google Apps Premier only - do not convert the file' +
                    ' on upload. (Else converts to native Google Docs format)')

If that means the --no-convert option only works with Google Apps Premier, 
which seems inconsistent with the new uploading of _any_ file type to standard 
Google Docs, it would make googlecl useless (and a big waste of time) to me.

Original comment by john.na...@gmail.com on 29 Jun 2010 at 1:22

[GoogleCodeExporter]

I suspect this is related to issue 199, especially since uploads used to work 
for you. See if "new-version" documents are enabled for you (and if they are, 
disable them), and try the most recent version in the trunk.

As for arbitrary uploads, The API seems to be lagging behind the functionality 
seen in the web interface: 
http://code.google.com/apis/documents/docs/3.0/developers_guide_protocol.html#Up
loadingArbitraryFileTypes (also note that is for v3 of the protocol, while we 
are still on v1 for compatibility and stability)

Original comment by tom.h.mi...@gmail.com on 12 Jul 2010 at 9:17

[GoogleCodeExporter]

I'm only interested in "arbitrary uploads", document types not supported by 
Google Docs that I just want to store in the cloud as backup, so it seems I'm 
SOL. This should be clearly documented so others don't waste as much time as I 
did.

Original comment by john.na...@gmail.com on 13 Jul 2010 at 12:18

[GoogleCodeExporter]

Issue 113 has been merged into this issue.

Original comment by tom.h.mi...@gmail.com on 20 Jul 2010 at 4:04

[GoogleCodeExporter]

Issue 158 is essentially the same problem as this, and contains a solution.

Original comment by tom.h.mi...@gmail.com on 31 Jul 2010 at 12:13

• Changed state: Duplicate

[GoogleCodeExporter]

tom:
It doesn't look like 158 is actually the same issue.  From what I can see 
(which may be totally wrong), issue 158 was resolved by pdf being added to the 
list of acceptable file types.  However, that doesn't help with those of us who 
want to be able to upload tgz files (my case), or, as this ticket's requester 
wanted, a kdb file.  Am I missing something?

Original comment by tuxg...@gmail.com on 31 Jul 2010 at 1:34

[GoogleCodeExporter]

Re Comment 7 by tuxgirl:
"Haste makes waste." In this case a waste of the effort put into this report.
No issue should be closed or merged unless and until the Reporter has a chance
to comment on the proposed action, and this issue should be immediately 
reopened.

Original comment by john.na...@gmail.com on 31 Jul 2010 at 2:08

[GoogleCodeExporter]

Sorry, I made a mistake. Let's summarize.

The original issue is that John (or anyone else, for that matter) cannot upload 
arbitrary file types to Google Docs through GoogleCL, even though it's possible 
through the web interface.

According to the API documentation at 
http://code.google.com/apis/documents/docs/3.0/developers_guide_protocol.html#Up
loadingArbitraryFileTypes users can only upload arbitrary file types if they 
are Google Apps Premier customers. It's likely that the POST request must also 
specify a gdata-version of 3. So this issue consists of several parts:

1) GoogleCL uses v1 of the gdata protocol.
2) Arbitrary upload types through the gdata API are only allowed for Google 
Apps Premier customers
3) The --no-convert option is not properly honored in googlecl

Issue 158 proposes a solution (hack) to #1. #2 is beyond my control (though you 
can voice your desire for a solution at 
http://code.google.com/p/gdata-issues/issues/detail?id=2073) 

The best I can do, specific to this issue, is remove the misleading 
--no-convert option. The most helpful solution would be to resolve Issue 158, 
then hope that arbitrary uploads for standard accounts are supported in the 
near future. That was my (admittedly poor) reasoning for merging the two issues.

Instead, I've created a new "open" label BlockedOnAPI. Is this to everyone's 
satisfaction?

Sorry again for the confusion.

Original comment by tom.h.mi...@gmail.com on 31 Jul 2010 at 3:23

• Changed state: BlockedOnAPI

[GoogleCodeExporter]

Issue 240 has been merged into this issue.

Original comment by tom.h.mi...@gmail.com on 31 Jul 2010 at 3:23

[GoogleCodeExporter]

Re Comment 9 by tom.h.mi...@gmail.com:

Thank you!

Original comment by john.na...@gmail.com on 1 Aug 2010 at 12:53

[GoogleCodeExporter]

Upgraded to latest current versions
Windows 7, Python 2.6.5, googlecl 0.9.9, gdata 2.0.11 final
now getting a different error:

C:\Python\Scripts>google docs upload PATHNAME
Traceback (most recent call last):
  File "google", line 577, in <module>
  File "google", line 571, in main
  File "google", line 374, in run_once
  File "googlecl\__init__.pyo", line 176, in read_access_token
  File "pickle.pyo", line 1370, in load
  File "pickle.pyo", line 858, in load
  File "pickle.pyo", line 966, in load_string
ValueError: insecure string pickle

Original comment by john.na...@gmail.com on 9 Aug 2010 at 10:36

[GoogleCodeExporter]

Solved the pickle problem as described here:
http://code.google.com/p/googlecl/issues/detail?id=204#c5

Now I get:
   >google docs upload --title FILE.rdf --folder BACKUP --no-convert PATH.rdf
   No supported filetype found for extension rdf
   Uploading as text/plain
   Loading PATH.rdf
   Upload success! Direct link: LINK

Result:
--folder option was honored
--no-convert seems to have been ignored
File was uploaded, but has a text icon, and can't be downloaded.

Original comment by john.na...@gmail.com on 11 Aug 2010 at 4:01

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

The error I get when I try to download:

>google docs get --title FILE.rdf --folder BACKUP FILE.rdf
Downloading FILE.rdf to FILE.rdf
Download of FILE.rdf failed: {'status': 500, 'body': '<!DOCTYPE html><html 
lang="en"> ...

No download file is created.

I do not get this error when I upload this same file using the Web interface 
and no conversion, and then download with googlecl:

>google docs get --title FILE.rdf --folder BACKUP FILE.rdf
>

(I can't find a downloaded file, but that's a different issue.)

Suggest this issue be changed to "docs upload --noconvert option not honored".

Original comment by john.na...@gmail.com on 11 Aug 2010 at 4:14

[GoogleCodeExporter]

John:
Are you saying that you are now able to upload files of any format via 
googlecl?  Have you tried a binary file?

I will attempt this afternoon to see if this problem is solved for my use case 
as well.

Original comment by tuxg...@gmail.com on 11 Aug 2010 at 8:15

[GoogleCodeExporter]

Re Comment 16 by tuxgirl:

While that prior test apparently worked because the contents of the file were 
text, a binary upload does still fail, presumably because the --no-convert 
option isn't being honored:

>google docs upload --title FILE.zip --folder BACKUP --no-convert FILE.zip
Loading FILE.zip
Failed to upload FILE.zip: {'status': 415, 'body': 'Content-Type 
application/zip is not a valid input type.', 'reason': 'Unsupported Media Type'}

Original comment by john.na...@gmail.com on 11 Aug 2010 at 10:00

[GoogleCodeExporter]

Okay.  In that case, this bug is not yet closed, in my opinion.  I believe that 
honoring the --noconvert option would resolve my complaint, though, so I am 
willing to alter it as suggested in comment 15 to "--noconvert option not 
honored"

Original comment by tuxg...@gmail.com on 11 Aug 2010 at 11:32

[GoogleCodeExporter]

Re Comment 18 by tuxgirl:
Agreed.

Original comment by john.na...@gmail.com on 11 Aug 2010 at 11:37

[GoogleCodeExporter]

The --no-convert option will be added in soon. The last couple of revisions 
have added support for the new Docs library, so once --no-convert is honored, 
Google Apps Premier users should be able to upload files of any type.

Just a reminder, though -- ONLY PREMIER accounts will be able to do arbitrary 
uploads. Any users following this issue that do not have a premier apps 
account, be sure to star 
http://code.google.com/p/gdata-issues/issues/detail?id=2073 and create a new 
issue in GoogleCL's issue tracker.

Original comment by tom.h.mi...@gmail.com on 12 Aug 2010 at 2:46

• Changed title: --no-convert option not honored
• Changed state: Started

[GoogleCodeExporter]

Re Comment 20 by tom.h.mi...@gmail.com:

WTF?!  How is this a "reminder"?  I saw nothing on this is the puff pieces and 
materials for arbitrary Google Docs files and googlecl, and have now wasted a 
great deal of time.  What did I miss and where is it?  Google needs to do a 
MUCH better job of communicating limitations like this, or risk alienating 
people like me.

Original comment by john.na...@gmail.com on 12 Aug 2010 at 3:33

[GoogleCodeExporter]

r374 looks at the --no-convert option. However, I need someone with an Apps 
Premier account to test uploading arbitrary files. Let me know (or check the 
FAQ, or ask the mailing list) if you need help running code from the repository.

As for John's complaint: It is rather difficult to find information on the 
limitations of the API until you actually start developing with it, which 
doesn't help the end user much.

The official references to this limitation that I've found are:
http://code.google.com/p/gdata-issues/issues/detail?id=2073
http://code.google.com/apis/documents/docs/3.0/developers_guide_protocol.html#Up
loadingArbitraryFileTypes

Outside of this thread (comments 3 and 9), the FAQ has a section "Why can't I 
upload XXX to Docs?" I'll add something to the main Manual once --no-convert 
has been proven to work for Premier users, and patch up the FAQ. To be honest, 
I didn't know about this limitation when I first started this project, and I 
apologize to anyone that was looking forward to a fully-functional 
--no-convert. 

Original comment by tom.h.mi...@gmail.com on 12 Aug 2010 at 4:13

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Thank you for another constructive and helpful response.
Please accept my apology for being so cranky.
My comment on issue 2073:
Google should at least promptly provide arbitrary file support in the API for 
those _paying_ for extra storage (but not Apps Premiere, which they may not 
need). I'm frankly dismayed and disappointed that it hasn't already done so.

Original comment by john.na...@gmail.com on 12 Aug 2010 at 4:22

[GoogleCodeExporter]

0.9.10 honors the --no-convert option for Premier users, but I can't test it 
since I don't have access to such an account. If anyone does, please test the 
use of that option and report back. Thanks!

Original comment by tom.h.mi...@gmail.com on 3 Sep 2010 at 2:21

[GoogleCodeExporter]

Original comment by tom.h.mi...@gmail.com on 8 Sep 2010 at 6:59

• Changed state: Feedback

[GoogleCodeExporter]

--no-convert apparently still broken as of 0.9.11 (with premium account):

$ google docs upload --no-convert --folder "Netværk" pdf/complete.pdf 
No supported filetype found for extension pdf
Uploading as text/plain
Loading pdf/complete.pdf
Failed to upload pdf/complete.pdf: {'status': 400, 'body': 'Could not convert 
document.', 'reason': 'Bad Request'}

$ google --version
google 0.9.11

Original comment by tais.han...@gmail.com on 19 Oct 2010 at 11:55

[GoogleCodeExporter]

Thanks for reporting back, tais.hansen!

It looks like you're using an early version of the gdata library. Could you 
upgrade to 2.0.12 and try again? Let me know if you need help with that. 

Original comment by tom.h.mi...@gmail.com on 19 Oct 2010 at 4:32

[GoogleCodeExporter]

I was using gdata-1.2.4 which is the newest package in Ubuntu Lucid.

I found gdata-2.0.8 released for Ubuntu Maverick a few days ago and rebuilt the 
package for Lucid. gdata-2.0.8 solves the issues I experienced.

Original comment by tais.han...@gmail.com on 19 Oct 2010 at 8:50

[GoogleCodeExporter]

Excellent, thanks for the quick feedback.

Marking this closed, and adding a caveat in the wiki Troubleshooting page.

Original comment by tom.h.mi...@gmail.com on 19 Oct 2010 at 9:17

• Changed state: Fixed