uwescience / sqlshare-pythonclient

The Python client for the SQLShare REST API
3 stars 1 forks source link

fetchdata.py #5

Closed mgavery closed 11 years ago

mgavery commented 11 years ago

Hi,

I am having trouble using fetchdata.py with a certain file. Other files download as expected. Thanks!

This works just fine: d-140-142-25-120:tools macoconnor$ python fetchdata.py -d "[che625@washington.edu].[BiGO_Methylation_oysterv9_GFF]" -f tsv -o /Users/macoconnor/Desktop/test.txt

This does not work: d-140-142-25-120:tools macoconnor$ python fetchdata.py -d "[sr320@washington.edu].[BiGill_methratio_v9_A.txt]" -f tsv -o /Users/macoconnor/Desktop/test.txt

This is the 'error' I get: Traceback (most recent call last): File "fetchdata.py", line 72, in main() File "fetchdata.py", line 66, in main data = fetchdata(args.sql, args.format) File "fetchdata.py", line 25, in fetchdata return conn.download_sql_result(sql, format) File "/Users/macoconnor/sqlshare-pythonclient/sqlshare/init.py", line 328, in download_sql_result return self.poll_selector(selector) File "/Users/macoconnor/sqlshare-pythonclient/sqlshare/init.py", line 252, in poll_selector raise SQLShareError("code: %s : %s" % (res.status, res.read())) sqlshare.SQLShareError: code: 400 : <?xml version="1.0" encoding="utf-8"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

Request Error

Request Error

The server encountered an error processing the request. Please see the service help page for constructing valid requests to the service. The exception message is 'A transport-level error has occurred when receiving results from the server. (provider: TCP Provider, error: 0 - An existing connection was forcibly closed by the remote host.)'. See server logs for more details. The exception stack trace is:

at RestWebRole.SSRestImpl.SqlSynchonousExecute(String sql) in C:\Users\UWCSE\Documents\Visual Studio 2010\Projects\New\SQLShareService\SQLShareService\RestWebRole\RestExecuteSql.cs:line 329 at RestWebRole.SSRestImpl.SqlToFileV1(String sql, String format, String filename) in C:\Users\UWCSE\Documents\Visual Studio 2010\Projects\New\SQLShareService\SQLShareService\RestWebRole\RestFileDownload.cs:line 39 at SyncInvokeSqlToFileV1(Object , Object[] , Object[] ) at System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke(Object instance, Object[] inputs, Object[]& outputs) at System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(MessageRpc& rpc) at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(MessageRpc& rpc) at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage31(MessageRpc& rpc) at System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean isOperationContextSet)

dhalperi commented 11 years ago

My guess is that this dataset is just freaking huge and that's causing some issues on the backend. But what you're downloading is just the raw data---not a query result---so why don't you just get the file here: http://eagle.fish.washington.edu/cnidarian/BiGill_methratio_v9_A.txt

mgavery commented 11 years ago

That is a real good point Dan, works for me :)

dhalperi commented 11 years ago

Thanks. I'll create a new issue to figure out why we can't download big files. Probably some sort of built-in timeout.