cloudsearchdomain upload-documents times out

[owocki]

On an Amazon Linux EC instance, trying to upload a sequence of JSON documents to cloudsearch:

aws cloudsearchdomain --endpoint-url http://url.to.2.cloudsearch.amazonaws.com upload-documents --content-type application/json --documents $file | tee -a upload.log

I get the following error

A server error (Unknown) occurred when calling the UploadDocuments operation: 

When running with --debug, I get the following stack trace:

2015-04-22 00:32:06,251 - MainThread - awscli.errorhandler - DEBUG - HTTP Response Code: 504
2015-04-22 00:32:06,251 - MainThread - awscli.clidriver - DEBUG - Exception caught in main()
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/awscli/clidriver.py", line 183, in main
    return command_table[parsed_args.command](remaining, parsed_args)
  File "/usr/local/lib/python2.7/dist-packages/awscli/clidriver.py", line 369, in __call__
    return command_table[parsed_args.operation](remaining, parsed_globals)
  File "/usr/local/lib/python2.7/dist-packages/awscli/clidriver.py", line 538, in __call__
    call_parameters, parsed_globals)
  File "/usr/local/lib/python2.7/dist-packages/awscli/clidriver.py", line 662, in invoke
    **parameters)
  File "/usr/local/lib/python2.7/dist-packages/botocore/client.py", line 191, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python2.7/dist-packages/botocore/client.py", line 242, in _make_api_call
    model=operation_model
  File "/usr/local/lib/python2.7/dist-packages/botocore/hooks.py", line 226, in emit
    return self._emit(event_name, kwargs)
  File "/usr/local/lib/python2.7/dist-packages/botocore/hooks.py", line 209, in _emit
    response = handler(**kwargs)
  File "/usr/local/lib/python2.7/dist-packages/awscli/errorhandler.py", line 70, in __call__
    http_status_code=http_response.status_code)
ServerError: A server error (Unknown) occurred when calling the UploadDocuments operation: 
2015-04-22 00:32:06,418 - MainThread - awscli.clidriver - DEBUG - Exiting with rc 255

Other info:

us-west-2 aws --version aws-cli/1.7.23 Python/2.7.6 Linux/3.13.0-48-generic

[jamesls]

Further up in the --debug log the raw http response from the server is shown (It will start with Response headers, then Response body:). Could you share that as well? That ideally has some sort of error message that will help.

Also, make sure that if you're referencing a file in --documents that you prefix it with file:// (http://docs.aws.amazon.com/cli/latest/userguide/cli-using-param.html#cli-using-param-file).

[owocki]

Hi @jamesls, it seems that if I just throttle 300 seconds per 20k rows I upload, the uploads work. Must be some sort of capacity issue on the server side. Thanks.

[jamesls]

Awesome, glad to hear you got it working.

[rafaelcmrj]

I know this issue is closed but I'm getting a timeout when I try to upload a 20mb XML file.

Read timeout on endpoint URL: "ENDPOINT_URL"