IOException / internal error(500/503) when doing stream inserts to BQ from web app running in Google App engine.

[GoogleCodeExporter]

What steps will reproduce the problem?
1. We are doing stream inserts to a table in Big query from our Google app 
engine web application.
2. Sometimes write to biq query fails giving 500/503 error code with 
IOException, details below. This is noticed for few times in our observation in 
the last 1 week. In 03.19.2016 this was noticed 9 times.

com.google.api.client.googleapis.json.GoogleJsonResponseException: 500
{
  "code": 500,
  "errors": [
    {
      "domain": "global",
      "message": "An internal error occurred and the request could not be completed.",
      "reason": "internalError"
    }
  ],
  "message": "An internal error occurred and the request could not be completed."
}

What is the expected output? What do you see instead?
We don't expect this internal errors(500/503) to happen. Pl let us know if you 
need further details to analyze the root cause for this IOException/errors. 

What version of the product are you using? On what operating system?
We are using BQ SDK google-api-services-bigquery-v2-rev154-1.18.0-rc.jar and 
are running our web application which writes to BQ in Google app engine.

Pl let us know any additional inputs/details/logs which might be needed to 
debug/ analyze this. Importantly, would like to know if this error should be 
expected and if so, suggest us on the suggested behavior when we encounter this 
error while write to BQ. 

Original issue reported on code.google.com by rajeshsr...@gmail.com on 21 Mar 2016 at 9:31

[GoogleCodeExporter]

You may see occasional errors when invoking tabledata.insertAll (caused by 
transient network issues, services restarting, etc). In those cases, you should 
retry the operation (ideally, after a short delay).

You can use an insertId to prevent duplicate inserts (as an insert may succeed, 
but still report failure):
https://cloud.google.com/bigquery/streaming-data-into-bigquery#dataconsistency

Don't forget to check 'insertErrors' in the response for partial success:
https://cloud.google.com/bigquery/troubleshooting-errors#streaming

If you can provide the full table ID where you are seeing errors, I can take a 
closer look. 

Original comment by ada...@google.com on 21 Mar 2016 at 8:21

• Changed state: Accepted

[GoogleCodeExporter]

Closing this issue, but feel free to reopen if you'd like to provide the table 
ID (I can check the logs for more specific information).

Original comment by ada...@google.com on 24 Mar 2016 at 5:20

• Changed state: Done