Closed torbjornvatn closed 7 years ago
Thanks for the report, and sorry for the issue.
Can you please check whether this works using the InProcessPipelineRunner
? If you are using an SDK newer than 1.6.0, you may want to switch. From the release notes for 1.6.0:
Added
InProcessPipelineRunner
, an improvement over theDirectPipelineRunner
that better implements the Dataflow model.InProcessPipelineRunner
runs on a user's local machine and supports multithreaded execution, unbounded PCollections, and triggers for speculative and late outputs.
In Dataflow 2.x, we've removed the old DirectPipelineRunner
and replaced it with the InProcessPipelineRunner
.
I've tried switching to InProcessPipelineRunner but got the exact same error I'm afraid.
We're writing all our BigQuery queries using the StandardSql option in the web console, but when I tried to execute a query from Dataflow running locally with this syntax enabled with
usingStandardSql()
I ran into this error:This is an example that triggers the error, and if I switch to
DataflowPipelineRunner
it works like a charm.