Error 500 (Unexpected) while performing a query via Java API

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Run a query using the BigQuery Java API
2.
3.

What is the expected output? What do you see instead?

It is expected a resultset but it returns the error: 500 OK
{
  "code" : 500,
  "errors" : [ {
    "domain" : "global",
    "message" : "Unexpected. Please try again.",
    "reason" : "internalError"
  } ],
  "message" : "Unexpected. Please try again."
}

What version of the product are you using? On what operating system?

To try solve the issue I have updated the API to latest API version, 1.19.0 but 
it returns the same error.

Please provide any additional information below.

The system working several hours ago.

Original issue reported on code.google.com by franci...@plusdigitalinstitute.com on 19 Aug 2014 at 7:40

[GoogleCodeExporter]

We are also getting the same error, using the Python API

Original comment by bas...@pagerduty.com on 19 Aug 2014 at 7:49

[GoogleCodeExporter]

same error with the Node API as well

Original comment by si...@rangle.io on 20 Aug 2014 at 3:44

[GoogleCodeExporter]

Thanks for the error report - just a couple of clarification questions:

- Is this happening for all of your query jobs or just a portion?
- Do you have affected job ids that you can post?
- At what time did the issue start?  Is it still ongoing?
- Does this just affect query RPCs, or other types of requests as well?

We had an issue between approximately 3 and 4pm PST (GMT-8) today that would 
have affected project.list API requests, but should not have affected query 
jobs.

I can successfully query BQ via an external account of my own; I don't see 
anything that immediately strikes me as unusual in our metrics.

Thank you,

Original comment by thomasp...@google.com on 20 Aug 2014 at 4:38

• Changed state: Accepted

[GoogleCodeExporter]

Hi,

We have an AppEngine App that pulls data out of BQ, but it started to fail 
early this morning PST.  

This is from AppEngine Log file

<HttpError 500 when requesting 
https://www.googleapis.com/bigquery/v2/projects/867761949342/queries?alt=json 
returned "Unexpected. Please try again.">
Traceback (most recent call last):
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1535, in __call__
    rv = self.handle_exception(request, response, e)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1529, in __call__
    rv = self.router.dispatch(request, response)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
    return route.handler_adapter(request, response)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1102, in __call__
    return handler.dispatch()
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 572, in dispatch
    return self.handle_exception(e, self.app.debug)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 570, in dispatch
    return method(*args, **kwargs)
  File "/base/data/home/apps/s~optimum-surface-468/2.377741363672781866/app/main.py", line 551, in get
    self._schedule_query()
  File "/base/data/home/apps/s~optimum-surface-468/2.377741363672781866/app/main.py", line 554, in _schedule_query
    self._persist(self.sql, self.batch_name)
  File "/base/data/home/apps/s~optimum-surface-468/2.377741363672781866/app/main.py", line 560, in _persist
    batch = BQ_CLIENT.batch(statements, cache=False)
  File "/base/data/home/apps/s~optimum-surface-468/2.377741363672781866/app/bigquery.py", line 65, in batch
    results = dict([[label, self.query(sql)] for label, sql in statements.iteritems()])
  File "/base/data/home/apps/s~optimum-surface-468/2.377741363672781866/app/bigquery.py", line 83, in query
    response = self._query(sql)
  File "/base/data/home/apps/s~optimum-surface-468/2.377741363672781866/app/bigquery.py", line 120, in _query
    result = self.service.jobs().query(projectId=self.project_number, body=config).execute(http=self.http)
  File "lib/oauth2client/util.py", line 132, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "lib/apiclient/http.py", line 723, in execute
    raise HttpError(resp, content, uri=self.uri)
HttpError: <HttpError 500 when requesting 
https://www.googleapis.com/bigquery/v2/projects/867761949342/queries?alt=json 
returned "Unexpected. Please try again.">

Original comment by bas...@pagerduty.com on 20 Aug 2014 at 5:23

[GoogleCodeExporter]

Is the issue still ongoing for you?  I see a slightly elevated rate of HTTP 
500s between 10 and 12 PST today, but nothing dramatic.

A few other questions that would help me troubleshoot:
- Is this happening for all of your query jobs or just a portion?
- Do you have affected job ids that you can post?
- Does this just affect query RPCs, or other types of requests as well?

Original comment by thomasp...@google.com on 20 Aug 2014 at 5:32

[GoogleCodeExporter]

Hey there, 

yes, it's still happening.

It seems to be happing to all of our queries.
We can't even get a job id, as it's failing when we submit the query job.  Is 
there a way to force a job id?
We are using the synchronous API (as documented for Python on this page, 
'Synchronous Query Examples': 
https://developers.google.com/bigquery/querying-data).

Original comment by bas...@pagerduty.com on 20 Aug 2014 at 5:57

[GoogleCodeExporter]

You can specify a job id when you submit a job, but if the job insert isn't 
coming back, then it's unlikely that this would help.  Specifying a job id is 
most useful if you want to be able to retry a job insert on failures - the 
system will reject a job with an id that already exists in the system.

Are you able to do job list or dataset list commands, for example?

Original comment by thomasp...@google.com on 20 Aug 2014 at 6:36

[GoogleCodeExporter]

Hi all,

Could you please post your project ids?  It would be very helpful to have a bit 
more information to troubleshoot this issue, thanks!

Original comment by thomasp...@google.com on 20 Aug 2014 at 7:38

[GoogleCodeExporter]

Hi - we're getting it on project IDs
regal-moment-665
plucky-hue-480

We're seeing this issue via a Chart.io query, and it is occurring on all 
queries using the API (but the same query done directly via web gui works fine)

Original comment by b...@redant.com.au on 20 Aug 2014 at 7:44

[GoogleCodeExporter]

> it is occurring on all queries using the API (but the same query done 
directly via web gui works fine)

Ah, that's an interesting clue.  Are you using the 
"https://www.googleapis.com/auth/bigquery.readonly" auth scope?  It looks like 
there was a problem in the API serving infrastructure that could have affected 
queries using that scope.

I'm looking into it further, but if you're still getting failures, could you 
please try using the  "https://www.googleapis.com/auth/bigquery" scope and let 
me know if that works?

Original comment by thomasp...@google.com on 20 Aug 2014 at 7:57

[GoogleCodeExporter]

I've synced up with some of my teammates.  The issue that was causing the 
bigquery.readonly scope to stop working should be resolved as of about 0.30 PDT 
(GMT-8) today (20 Aug).  The bigquery (read/write) scope should have been 
unaffected, so please try that scope if you're still getting errors.

We will be working with the API serving team to ensure that the 
bigquery.readonly scope can continue to be used going forward.

Please let me know if this helps; I'll check in on this issue again first thing 
in the morning.

Best,

Original comment by thomasp...@google.com on 20 Aug 2014 at 8:15

[GoogleCodeExporter]

Thankyou! All working fine (for me anyway) now

Original comment by b...@redant.com.au on 20 Aug 2014 at 10:17

[GoogleCodeExporter]

Now it's working fine for me too.

Original comment by franci...@plusdigitalinstitute.com on 20 Aug 2014 at 11:01

[GoogleCodeExporter]

Fantastic, thank you for the confirmation.  Resolving issue.

Original comment by thomasp...@google.com on 20 Aug 2014 at 5:22

• Changed state: Fixed