search

karthik / rDrop

Programmatic interface to Dropbox
65 stars 17 forks source link

dropbox_save() does not handle large files well #1

Open karthik opened 12 years ago

karthik commented 12 years ago 
df <- data.frame(x = rnorm(100000)) 
dropbox_save(token, df, file = "large_data.rda") 
Error in fromJSON(OAuthRequest(cred, url, , "PUT", upload = TRUE, readfunction = input,  : 
  error in evaluating the argument 'content' in selecting a method for function 'fromJSON': Error in function (type, msg, asError = TRUE)  : Empty reply from server
Calls: OAuthRequest ... oauthCommand -> curlPerform -> .Call -> <Anonymous> -> fun
Calls: dropbox_save -> fromJSON
randywreed commented 10 years ago

This may be related to issue. If the mistake is my syntax please correct me. When I execute a dropbox_save of a large file (1.3gb) I get the following:

dropbox_save(dropbox_credentials, GSS.2012.CS.df, file="GSS_all.rdata") Error in file(con, "r") : cannot open the connection

A smaller file 131mb also gets an error

dropbox_save(dropbox_credentials, gss2008_12, file="GSS_2008_12_All_Fields.rdata") Error in fromJSON(OAuthRequest(cred, url, , "PUT", upload = TRUE, readfunction = input, : error in evaluating the argument 'content' in selecting a method for function 'fromJSON': Error in function (type, msg, asError = TRUE) : GnuTLS recv error (-9): A TLS packet with unexpected length was received.

a third file 3.4mb gets the same error

dropbox_save(dropbox_credentials, gss2008_12.relcombine, file="gss2008_12.rdata") Error in fromJSON(OAuthRequest(cred, url, , "PUT", upload = TRUE, readfunction = input, : error in evaluating the argument 'content' in selecting a method for function 'fromJSON': Error in function (type, msg, asError = TRUE) : GnuTLS recv error (-9): A TLS packet with unexpected length was received.

karthik commented 10 years ago

Hi @randywreed Looking into this. Will get back to you shortly.

duncantl commented 10 years ago

Just to chime in here with very limited time... The first of these 'Error in file(con, "r") : cannot open the connection' looks like a local issue and at a very quick guess that the file named is misspelled. The "r" mode indicates this.

The GnuTLS errors are curious, but it would be really helpful to have a verbose output from a request, i.e., create a curlHandle() object with verbose = TRUE and then use this for all requests so we can see the output. Often (but not always), such errors are local network issues.

justinlent commented 10 years ago

I'm also seeing this error (but without the others mentioned previously). All I'm trying to do is save a pretty small dataframe. dim(DF) = 2863 6

Error in fromJSON(OAuthRequest(cred, url, , "PUT", upload = TRUE, readfunction = input, : error in evaluating the argument 'content' in selecting a method for function 'fromJSON': Error in function (type, msg, asError = TRUE) : Empty reply from server